-
Couldn't load subscription status.
- Fork 41
This fixes issue #58. #59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| } catch (IOException e) { | ||
| throw new IOException(EMPTY_INPUT_STREAM); | ||
| int bytesLeft = sasFileProperties.getHeaderLength() - currentFilePosition; | ||
| int skipped = sasFileStream.skipBytes(bytesLeft); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is a bit risky -- in some rare cases this would skip less bytes than it was asked to, because of network latency for example, so ideally this should be something like skipFully in IOUtils. As we don't really want to bring another dependency, probably it makes sense to copy IOUtils' logic (without some IFs, etc, of course). Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, just seeing this now. Yes, I completely agree.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like DataInputStream's skipBytes does do the while...loop so that's not a problem...I don't think.
What is a problem is that FileInputStream can lie about bytes skipped, so y, I'll copy and paste from IOUtils.skipFully().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the decompiler in Java 8 on DataInputStream
public final int skipBytes(int n) throws IOException {
int total = 0;
int cur = 0;
while ((total<n) && ((cur = (int) in.skip(n-total)) > 0)) {
total += cur;
}
return total;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I don't like about the current version is that if readFully fails, then eof=true and the user will get a null on next row instead of an EOFException, but if skip fails, then there's an exception.
Any preference/recommendation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, agree, so I see the reason in your suggested change, so I would avoid readFully in favor of IOUtils.skipFully (it's just better than streams' skip/read methods) so perhaps just copy and paste those IOUtils methods as the licence is the same?
|
Improved this behaviour as a part of #82 |
I had to make some local modifications to get the unit tests to work, but those are for another issue.
Let me know if this works, and thank you!