Cache Parse() results #26
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Each parse takes ~2ms on my machine, and it's pretty common throughout the life of a running
process, to parse identical user-agent strings. This adds a very primitive cache similar in vein
to the cache inside the `urlparse` package.
Before:
```
$ python -m timeit -s 'from ua_parser.user_agent_parser import Parse' 'Parse("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.52 Safari/537.36")'
100 loops, best of 3: 2.14 msec per loop
```
After:
```
$ python -m timeit -s 'from ua_parser.user_agent_parser import Parse' 'Parse("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.52 Safari/537.36")'
1000000 loops, best of 3: 0.956 usec per loop
```
| @@ -205,12 +209,20 @@ def Parse(user_agent_string, **jsParseBits): | |||
| A dictionary containing all parsed bits | |||
| """ | |||
| jsParseBits = jsParseBits or {} | |||
| return { | |||
| key = (user_agent_string, repr(jsParseBits)) | |||
There was a problem hiding this comment.
I'm not happy with using repr() like this, but we need a hashable type to use as a key.
mattrobenolt
added a commit
to getsentry/sentry
that referenced
this pull request
Oct 11, 2015
See upstream patch: ua-parser/uap-python#26
selwin
pushed a commit
to selwin/uap-python
that referenced
this pull request
Oct 14, 2015
updating ua object to match implementations
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Each parse takes ~2ms on my machine, and it's pretty common throughout the life of a running
process, to parse identical user-agent strings. This adds a very primitive cache similar in vein
to the cache inside the
urlparsepackage.Before:
After:
Cache memory overhead:
Given the user agent of
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.52 Safari/537.36, we get280 bytesper parsed object.So that means a
MAX_CACHE_SIZEof 20, will incur an overhead of 5600 bytes. Granted, this is also ignoring the cache key used in the_parsed_cachedict, but we're in the ballpark of a few KB total at most. :)