Just sharing results of some comparisons that I ran, no issues here.
The great sklearn package has recently added an LDA implementation which is easy to use with sklearn pipelines. It was interesting to compare it to gensim. This is based on @chyikwei 's test script. Thanks a lot for sharing it!
Sklearn and gensim basically agree, only one minor issue found. Results of comparison are in this spreadsheet.
Validation method. If perplexities are within 0.1% then I wouldn't worry, the implementations are the same to me.
The perplexity bounds are not expected to agree exactly here because bound is calculated differently in gensim vs sklearn. Sklearn initialises doc_topic_distr aka q(theta|gamma) as np.ones while gensim uses random.gamma. Also sklearn using cython which creates numerical 6th dp differences.
Results.
Batch mode. With decay = 0, the two implementations agree. Tested 5 M-steps on 10k documents.
Mini-batch mode aka 'online'. Single core gensim LDA and sklearn agree up to 6dp with decay =0.5 and 5 M-steps.
However comparing multicore LDA and gensim is harder as the parallelisation models are different.
Gensim is fully async as in this blog post while sklearn doesn't go that far and parallelises only E-steps. They agree after figuring out right parameters to run with same number of docs per thread in each, see spreadsheet for single M-step test results.
Performance
Sklearn with cython is 3 times faster than gensim due to using cython.
After re-writing sklearn's cython functions in Python I got a 2x slow down.
The test code is in this gist. The heavily logged versions of sklearn and gensim are here.. They are good to make sure intermediate steps agree in BeyondCompare
Just sharing results of some comparisons that I ran, no issues here.
The great sklearn package has recently added an LDA implementation which is easy to use with sklearn pipelines. It was interesting to compare it to gensim. This is based on @chyikwei 's test script. Thanks a lot for sharing it!
Sklearn and gensim basically agree, only one minor issue found. Results of comparison are in this spreadsheet.
Validation method. If perplexities are within 0.1% then I wouldn't worry, the implementations are the same to me.
The perplexity bounds are not expected to agree exactly here because bound is calculated differently in gensim vs sklearn. Sklearn initialises doc_topic_distr aka q(theta|gamma) as np.ones while gensim uses random.gamma. Also sklearn using cython which creates numerical 6th dp differences.
Results.
Batch mode. With decay = 0, the two implementations agree. Tested 5 M-steps on 10k documents.
Mini-batch mode aka 'online'. Single core gensim LDA and sklearn agree up to 6dp with decay =0.5 and 5 M-steps.
However comparing multicore LDA and gensim is harder as the parallelisation models are different.
Gensim is fully async as in this blog post while sklearn doesn't go that far and parallelises only E-steps. They agree after figuring out right parameters to run with same number of docs per thread in each, see spreadsheet for single M-step test results.
Performance
Sklearn with cython is 3 times faster than gensim due to using cython.
After re-writing sklearn's cython functions in Python I got a 2x slow down.
The test code is in this gist. The heavily logged versions of sklearn and gensim are here.. They are good to make sure intermediate steps agree in BeyondCompare