Backport filebeat registry state update improvements #7188
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Backport changes from elastic#6346 and elastic#6347, to improve state update performance on large registry files. - make States type easier to find by moving into separate states.go file - Add ID->array index, index for faster lookups on update and find operations. When updating states in a big registry, the Updates converged to quadratic complexity. The index helps in keeping the complexity about linear in number of state updates. - Do not gc state if there are no 'pending' entries
houndci-bot
reviewed
May 28, 2018
|
|
||
| // State is used to communicate the reading state of a file | ||
| type State struct { | ||
| Id string `json:"-"` // local unique id for comparison and indexing states. |
houndci-bot
reviewed
May 28, 2018
|
|
||
| // State is used to communicate the reading state of a file | ||
| type State struct { | ||
| Id string `json:"-"` // local unique id for comparison and indexing states. |
Author
|
Closing, as backport is not working as stable as I did hope for. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport changes from #6346 and #6347, to improve state update
performance on large registry files.
operations. When updating states in a big registry, the Updates
converged to quadratic complexity. The index helps in keeping the
complexity about linear in number of state updates.