Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add real-world BOM utf tests & add ZTransducer#utfDecode to smartly use BOM decoders #3980

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 28, 2020
Merged

Add real-world BOM utf tests & add ZTransducer#utfDecode to smartly use BOM decoders #3980

merged 1 commit into from
Jul 28, 2020

Conversation

@er1c
Copy link
Contributor

@er1c er1c commented Jul 17, 2020 •
edited
Loading

Based upon #3981 - will rebase once it is merged.

  • Add ZTransducer.utfDecode to auto-detect byte order marks
  • Add ZTransducer.utf32Decode
  • Add ZTransducer.utf32BEDecode
  • Add ZTransducer.utf32LEDecode
  • Add real-world BOM and utf tests for the ZTransducer UTF decoders

@er1c er1c force-pushed the bom branch 4 times, most recently from 9145f12 to f8473eb Compare July 17, 2020 21:23
@er1c er1c changed the title Add real-world BOM and utf tests for the ZTransducer UTF decoders Add real-world BOM utf tests & add ZTransducer#utfDecode to smartly use BOM decoders Jul 17, 2020
@er1c er1c marked this pull request as ready for review July 17, 2020 21:29
@er1c er1c requested a review from iravid as a code owner July 17, 2020 21:29
@er1c
Copy link
Contributor Author

er1c commented Jul 17, 2020

@mschuwalow mind taking a look at this?

@er1c er1c mentioned this pull request Jul 18, 2020
Closed
7 tasks
@iravid
Copy link
Member

iravid commented Jul 20, 2020

Love the additional coverage and the utfDecode transducer. Can you rebase the branch?

@er1c
Copy link
Contributor Author

er1c commented Jul 20, 2020

@iravid already done :) waiting on CI

@er1c er1c force-pushed the bom branch 5 times, most recently from 1e44410 to 43d9b08 Compare July 21, 2020 03:00
@er1c
* Add ZTransducer.utfDecode to auto-detect byte order marks
ac64bce 
* Add ZTransducer.utf32Decode
* Add ZTransducer.utf32BEDecode
* Add ZTransducer.utf32LEDecode
* Add real-world BOM and utf tests for the ZTransducer UTF decoders
@er1c
Copy link
Contributor Author

er1c commented Jul 21, 2020

@iravid status is green. I guess the biggest question here is the behavior for when the UTF_32 BOMs aren't supported by Charset

iravid
iravid approved these changes Jul 28, 2020
View reviewed changes
Copy link
Member

@iravid iravid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay here @er1c! Fantastic contribution. Thanks!

@iravid iravid merged commit d278621 into zio:master Jul 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iravid iravid iravid approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@er1c @iravid