Bug with Japanese Kanji support #2403
Description
Discussed in https://github.com/meilisearch/meilisearch/discussions/2391
Originally posted by toomozoo May 13, 2022
Thank you for supporting Japanese in the 0.27.0 version!
I have tried several Kanji searches and did not get the results I wanted.
Do I need to do any settings to search in Japanese?
% brew install meilisearch
% meilisearch -V
meilisearch-http 0.27.0
% irb
require 'json'
require 'meilisearch'
MeiliSearch::VERSION
=> "0.18.3"
client = MeiliSearch::Client.new('http://127.0.0.1:7700')
json = JSON.parse('[{"id": "1","name": "東京バナナ"},{"id": "2","name": "東京 ポテチ"}]')
client.index('test').add_documents(json)
# NG
client.index('test').search("東")
=> {"hits"=>[], "nbHits"=>0, "exhaustiveNbHits"=>false, "query"=>"東", "limit"=>20, "offset"=>0, "processingTimeMs"=>0}
# NG
client.index('test').search("東京")
=> {"hits"=>[], "nbHits"=>0, "exhaustiveNbHits"=>false, "query"=>"東京", "limit"=>20, "offset"=>0, "processingTimeMs"=>0}
# OK
client.index('test').search("バ")
=> {"hits"=>[{"id"=>"1", "name"=>"東京バナナ"}], "nbHits"=>1, "exhaustiveNbHits"=>false, "query"=>"バ", "limit"=>20, "offset"=>0, "processingTimeMs"=>0}
```</div>
---
## TODO
- [ ] Implement changes in [Milli](https://github.com/meilisearch/milli/): https://github.com/meilisearch/meilisearch/issues/3357
- [ ] Release a Milli version containing these changes
- [ ] Bump this new Milli version in Meilisearch and merge it into `main`