tolower() did not handle characters U+023A and U+023E #1406
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This call assert_equal("IÌÍÎÏĨĪĬĮǏỈ", tolower("iìíîïĩīĭįǐỉ")) I'm not sure why yet. I'll investigate, but so far I could not reproduce the issue on my machine :-( |
|
Dominique Pellé wrote:
This call assert_equal("IÌÍÎÏĨĪĬĮǏỈ", toupper("iìíîïĩīĭįǐỉ"))
failed in CI on some targets and succeeded on others.
I'm not sure why yet. It looks like a bug that existed prior to my
patch, but bug was revealed by the new test, since I managed to
reproduce a failure there locally with vim-8.0.205 without my patch
but with this new test.
This is how it failed without my patch but with the new test:
```
From test_alot.vim:
Found errors in Test_tolower():
function RunTheTest[13]..Test_tolower line 20: Expected 'iìíîïĩīĭįiǐỉ' but got 'iìíîïĩīĭįİǐỉ'
function RunTheTest[13]..Test_tolower line 70: Expected 'ⱥ ⱦ' but got 'Ⱥ Ⱦ'
```
The test failure at line 70 without my patch is expected since this is what my patch fixes.
But the test failure at line 20 is not expected.
I'll investigate soon.
Perhaps the 'casemap' option helps?
…--
Vim is like Emacs without all the typing. (John "Johann" Spetz)
/// Bram Moolenaar -- [email protected] -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
|
Regression which cause test failire was introduced by my previous checkin.
|
@brammool wrote:
No. My patch simply had a bug, introduced in the last minute |
chrisbra
pushed a commit
to chrisbra/vim
that referenced
this pull request
Mar 25, 2017
Problem: When making a character lower case with tolower() changes the byte
cound, it is not made lower case.
Solution: Add strlow_save(). (Dominique Pelle, closes vim#1406)
bfredl
added a commit
to bfredl/neovim
that referenced
this pull request
Apr 9, 2017
Problem: When making a character lower case with tolower() changes the byte
cound, it is not made lower case.
Solution: Add strlow_save(). (Dominique Pelle, closes vim/vim#1406)
vim/vim@cc5b22b
Join almost identical strup_save and strlow_save functions to one
Function.
bfredl
added a commit
to bfredl/neovim
that referenced
this pull request
Apr 9, 2017
Problem: When making a character lower case with tolower() changes the byte
cound, it is not made lower case.
Solution: Add strlow_save(). (Dominique Pelle, closes vim/vim#1406)
vim/vim@cc5b22b
Join almost identical strup_save and strlow_save functions to one
Function.
bfredl
added a commit
to bfredl/neovim
that referenced
this pull request
Apr 9, 2017
Problem: When making a character lower case with tolower() changes the byte
cound, it is not made lower case.
Solution: Add strlow_save(). (Dominique Pelle, closes vim/vim#1406)
vim/vim@cc5b22b
Join almost identical strup_save and strlow_save functions to one
Function.
bfredl
added a commit
to bfredl/neovim
that referenced
this pull request
Apr 9, 2017
Problem: When making a character lower case with tolower() changes the byte
cound, it is not made lower case.
Solution: Add strlow_save(). (Dominique Pelle, closes vim/vim#1406)
vim/vim@cc5b22b
Join almost identical strup_save and strlow_save functions to one
Function.
bfredl
added a commit
to bfredl/neovim
that referenced
this pull request
Apr 10, 2017
Problem: When making a character lower case with tolower() changes the byte
cound, it is not made lower case.
Solution: Add strlow_save(). (Dominique Pelle, closes vim/vim#1406)
vim/vim@cc5b22b
Join almost identical strup_save and strlow_save functions to one
Function.
desvp
pushed a commit
to desvp/vim
that referenced
this pull request
May 30, 2017
Problem: When making a character lower case with tolower() changes the byte
cound, it is not made lower case.
Solution: Add strlow_save(). (Dominique Pelle, closes vim#1406)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Function tolower() did not handle correctly characters that have different
number bytes in utf8 sequence after they get transformed into lowercase.
For example, characters Ⱥ (U+023A) or Ⱦ (U+023E) have 2 bytes
utf8 sequences in upper cases, and 3 bytes in lower case.
Before this fix
:echo tolower("Ⱥ")
Ⱥ (wrong!)
After this fix:
:echo tolower("Ⱥ")
ⱥ (correct)
Tests are also added for tolower() and toupper().