Thank you so much for the information. I managed to achieve what I wanted to use the DB Browser for SQLite.

@woaidangyang

Would you mind sharing with me how you did that? I downloaded the software you mentioned, but I'm still at a loss as to what to do exactly.

@xxyzz

Would you please consider adding an option to enable all the queries?

Thank you.

I added two buttons in this commit: af2a438

And you could download the test plugin from here: https://github.com/xxyzz/WordDumb/actions/runs/7554770944

If all the Korean words can be recognized, Worddumb will be a game-changer for reading Korean books on Kindle. Kindle does not support Korean dictionaries(looking up Korean words) even though it supports the language.

I looked at the previous discussions you provided. A customized phrasematcher is to hard for a Python novice like me. But since 봄 and 결정 can be recognized in the previous examples, isn't Spacy's model a bit better than the default one? I don't quite understand why that will need all forms of lemma. And the SQL file does contain a lot of forms when I opened it with an SQL browser.

Here is a possible test of the two models. Below is a paragraph of Korean

초안을 다 완성해가는 지금, 제가 가장 걱정하고 있는 것은 과연 누가 이 기록을 읽게 될 것인가 하는 문제입니다. 이 글의 목적은 그 마을의 존재를 최대한 많은 사람에게 알리는 것일까요. 아니면 기록하고 남기는 것일까요. 초안을 처음 쓰기 시작했을 때 저는 후자를 생각했습니다. 학술적인 이유로, 기록만을 목적으로 글을 완성하는 것도 충분히 의미 있다고요. 하지만 지금은 이 이야기를 많은 사람이 읽어주었으면 좋겠다는 생각이 들어요. 소수의 사람들에게만 기억되는 것은, 제 연구 대상들만으로 충분하지 않나 생각하고요.

Wordwise can only recognize six words in bold(지금, 가장, 과연, 최대한, 처음, 연구, all of them appear independently with a space before and after) with all the lemmas enabled and Kindle set to show most hints.

The other words in Italics(not showing Italics now, used bold instead) are not captured because of the particles issue. Here are the original forms of words that failed.

초안 draft n.
완성하다 complete v.
저 I
걱정하다 worry v.
기록하다 record v. (기록 record n.)
읽다 read v.
문제 issue n.
글 article n.
목적 purpose n.
마을 village n.
존재 existence n.
많다 many adj.
사람 people n.
알리다 tell v.
남기다 keep v.
쓰다 write v.
시작하다 start v.
후자 the latter n.
생각하다 think v.
학술적 academic adj.
이유 reason n.
충분히 sufficiently adj.
의미 meaning n.
이야기 story n.
소수 minority n.
기억되다 be remembered v. (기억하다 remember)
대상 subject, target n.
충분하다 enough adj.

I have omitted some of the words that appear more than once.

Even if just all the nouns are captured, it will be a great improvement. With some simple rules, maybe the verbs and adjectives that all end with 다 can be recognized as well. I wonder if Spacy's model will perform better.

Sorry I don't think it is feasible to use customized lemmatizer nor phase matcher. This probably will have to wait for spacy improving their lemmatizer in future release.

I can't get the mecab lemmatizer to work with spacy's default Korean nlp class, and I'm not sure if the trained model could change lemmatizer. I also don't know how to implement phase matcher, and because spacy model breaks the characters, it's difficult to know the token location.

I forgot to say you could disable the "use pos" option, spacy model's lemmatizer is not used when this option is disabled and if a word has its lemma or inflection forms in the word wise db it will be added.

Korean forms extracted from Wiktionary could be matched by default currently on master branch. Results are mostly the same as disable "use POS" option in previous versions, this option is now removed.

Korean forms extracted from Wiktionary could be matched by default currently on master branch. Results are mostly the same as disable "use POS" option in previous versions, this option is now removed.

Thank you so much. It has been some time since I used this plugin on Korean files. I will try and see if the result improves.