Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Encountering issue with scanning larger datasets - TypeError in dbstore.py #66

@weloewe

Description

@weloewe

With larger file counts (50M, e.g.) encountering an issue with TypeError. This is not seen with small counts (14M, e.g.). Is there a a workaround for this?

This is running with 32 processes "fprof $DIR" and seeing:
Traceback (most recent call last):
File "/home/loewe/packages/pcircle-py3.9/bin/fprof", line 8, in <module>
sys.exit(main())
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/fprof.py", line 768, in main
circle.begin(treewalk)
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/circle.py", line 224, in begin
self.loop()
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/circle.py", line 257, in loop
self.task.process()
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/fprof.py", line 397, in process
self.handle_file_or_dir(spath, st)
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/fprof.py", line 487, in handle_file_or_dir
self.process_dir(spath, st)
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/fprof.py", line 328, in process_dir
self.circle.enq(entry.path)
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/circle.py", line 275, in enq
self.workq_init(self.dbname, G.resume)
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/circle.py", line 194, in workq_init
self.workq_db = DbStore(self.dbname, resume=G.resume)
File "/home/loewe/treewalk-py3/local/miniconda3/envs/pcircle-py3.9/lib/python3.9/site-packages/pcircle/dbstore.py", line 85, in __init__
(sqlite3.Binary(str(0)),))
TypeError: memoryview: a bytes-like object is required, not 'str'

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions