Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ignore bson unicode decode error #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

ignore bson unicode decode error #127

wants to merge 1 commit into from

Conversation

islue
Copy link
Contributor

@islue islue commented May 11, 2017

I met such an error and the backup process hung.

[2017-05-11 04:18:47,505] [CRITICAL] [MongodumpThread-5] [Oplog:load:55] Error reading oplog file /var/lib/mongodb-consistent-backup/mytable/20170511_0256/ps-rs1/dump/oplog.bson! Error: 'utf8' codec can't decode byte 0xa1 in position 0: invalid start byte
[2017-05-11 04:18:47,505] [ERROR] [MongodumpThread-5] [MongodumpThread:run:121] Error loading oplog: 'utf8' codec can't decode byte 0xa1 in position 0: invalid start byte
Traceback (most recent call last):
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.0.1-py2-none-any.whl.50d2483f9d3c33ae9bb528471e56ca495b2758f3/mongodb_consistent_backup-1.0.1-py2-none-any.whl/mongodb_consistent_backup/Backup/Mongodump/MongodumpThread.py", line 119, in run
    oplog.load()
  File "/var/lib/mongodb-consistent-backup/.pex/install/mongodb_consistent_backup-1.0.1-py2-none-any.whl.50d2483f9d3c33ae9bb528471e56ca495b2758f3/mongodb_consistent_backup-1.0.1-py2-none-any.whl/mongodb_consistent_backup/Oplog/Oplog.py", line 56, in load
    raise OperationError(e)
OperationError: 'utf8' codec can't decode byte 0xa1 in position 0: invalid start byte

I'm not sure if it is caused by invalid bson data, but it corrupted the backup anyway. Since the codes show only 'ts' is concerned, I suppose that ignoring the error is ok. How to you think?

I ran the patched version and it worked for me.

References:
https://jira.mongodb.org/browse/PYTHON-721

@timvaillancourt timvaillancourt mentioned this pull request May 11, 2017
Merged
@timvaillancourt
Copy link
Contributor

timvaillancourt commented May 11, 2017 •
edited
Loading

Thanks @islue, this is a good idea. I pulled this into our 1.0.2 branch under PR #130.

After successful testing of 1.0.2 it will become 'master' so I will close this PR now.

@islue islue deleted the unicode_decode_error_handler branch May 12, 2017 02:45
@timvaillancourt timvaillancourt mentioned this pull request May 15, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
1 more reviewer

@timvaillancourt timvaillancourt timvaillancourt approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@islue @timvaillancourt