Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

Wumpf
Copy link
Member

@Wumpf Wumpf commented Aug 22, 2025

Related

What

previous PR was too aggressive: made the viewer "deaf" to anything from the SDK after closing one SDK originated recording

@Wumpf Wumpf requested a review from jprochazk August 22, 2025 09:17
@Wumpf Wumpf added the exclude from changelog PRs with this won't show up in CHANGELOG.md label Aug 22, 2025
Copy link

github-actions bot commented Aug 22, 2025

Web viewer built successfully. If applicable, you should also test it:

  • I have tested the web viewer
Result Commit Link Manifest
4325fd5 https://rerun.io/viewer/pr/10969 +nightly +main

Note: This comment is updated whenever you push a commit.

@jprochazk
Copy link
Member

I think this one is still too aggressive, the only thing we should not retain are redap connections. Nothing else is 1 connection == 1 recording

Copy link
Member

@jprochazk jprochazk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than that LGTM

Comment on lines +571 to +588
let should_close = match &data_source {
// Specific files should stop streaming when closing them.
SmartChannelSource::File(_) => true,

// Specific HTTP streams should stop streaming when closing them.
SmartChannelSource::RrdHttpStream { .. } => true,

// Specific GRPC streams should stop streaming when closing them.
// TODO(#10967): We still stream in some data after that.
SmartChannelSource::RedapGrpcStream { .. } => true,

// Don't close generic connections (like to an SDK) that may feed in different recordings over time.
SmartChannelSource::RrdWebEventListener
| SmartChannelSource::JsChannel { .. }
| SmartChannelSource::Sdk
| SmartChannelSource::Stdin
| SmartChannelSource::MessageProxy(_) => false,
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let should_close = match &data_source {
// Specific files should stop streaming when closing them.
SmartChannelSource::File(_) => true,
// Specific HTTP streams should stop streaming when closing them.
SmartChannelSource::RrdHttpStream { .. } => true,
// Specific GRPC streams should stop streaming when closing them.
// TODO(#10967): We still stream in some data after that.
SmartChannelSource::RedapGrpcStream { .. } => true,
// Don't close generic connections (like to an SDK) that may feed in different recordings over time.
SmartChannelSource::RrdWebEventListener
| SmartChannelSource::JsChannel { .. }
| SmartChannelSource::Sdk
| SmartChannelSource::Stdin
| SmartChannelSource::MessageProxy(_) => false,
};
let should_close = matches!(data_source, SmartChannelSource::RedapGrpcStream { .. });

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't want to use matches since it doesn't warn on new sources

@Wumpf
Copy link
Member Author

Wumpf commented Aug 22, 2025

Nothing else is 1 connection == 1 recording

Why would it not be the case for http and files?
Http is the reason why we had the exact same failure (plus the assert issue I previously linked) easily repro on the flight data example

@jprochazk
Copy link
Member

jprochazk commented Aug 22, 2025

Why would it not be the case for http and files?

Because the .rrd format supports concatenating (not merging) streams into a single file. So in both of those cases, you could be served an .rrd file which contains 2 or more logical recordings. Nothing is stopping an .rrd file from having more than one SetStoreInfo message. That means if you're streaming that file over HTTP or reading it from disk, and you close it, then it's possible that at some point in the stream it creates and begins populating a new store. So the reasoning for SDK, stdin, messageproxy, etc. should be the same as for http and files

@jprochazk
Copy link
Member

jprochazk commented Aug 22, 2025

I haven't tried to understand what is up with the other issue the previous PR fixed yet, maybe there's a different way to fix it? Also, hint hint nudge nudge cough #8631 cough

@Wumpf
Copy link
Member Author

Wumpf commented Aug 22, 2025

in a nutshell we have the exact same issue you fixed for the grpc entries also with streaming from http: if you close a recording that streams from file or http, we also have to close the data sources otherwise recordings will show up again. In Debug this causes a debug assertion to trigger on top of that which your original PR fixes (and this PR tries to preserve by means of still removing those sources)

Also, hint hint nudge nudge cough #8631 cough

lalalala I can't hear you ;-) (in this pr at least 😄 )

@Wumpf
Copy link
Member Author

Wumpf commented Aug 22, 2025

[....] So the reasoning for SDK, stdin, messageproxy, etc. should be the same as for http and files

that does make sense :/ except that it's quite the corner case. I think it's fair to say that the close on those recordings also closes the connection? It's a bit odd that we have this as a separate concept internally that we don't expose :(
I.e. I think the bug of not receiving more recordings is better than the ghost recordings that can come into existance right now (plus the annoying debug assertion described in #9848)

@jprochazk
Copy link
Member

jprochazk commented Aug 22, 2025

I think the bug of not receiving more recordings is better than the ghost recordings that can come into existance right now

I think I agree, and it's an edge case that should eventually go away. Seems acceptable to tell people "don't remove the data source if you have more than one logical recording in it and you want to see both". It seems unlikely for people to even do that in the first place

@Wumpf
Copy link
Member Author

Wumpf commented Aug 23, 2025

👍 but good you brought up this concern - I didn't really consider the possibility. We're clearly not "done" here!

@Wumpf Wumpf merged commit bf92bc8 into main Aug 25, 2025
39 checks passed
@Wumpf Wumpf deleted the andreas/fix-remove-dataset-rx-fix branch August 25, 2025 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exclude from changelog PRs with this won't show up in CHANGELOG.md
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants