Add option to exclude lines from sample log files in pipeline tests #380
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
💔 Build Failed
Expand to view the summary
Build stats
Trends 🧪Steps errors
Expand to view the steps failures
|
mtojek
reviewed
Jun 21, 2021
| ```yml | ||
| multiline: | ||
| first_line_pattern: "^(?:[0-9]{1,3}\\.){3}[0-9]{1,3}" | ||
| exclude_lines: |
There was a problem hiding this comment.
I think you should prepare a PR for the package-spec first.
| The `multiline` section ([raw files](#raw-files) only) configures the log file reader to correctly detect multiline log entries using the `first_line_pattern`. Use this property if your logs may be split into multiple lines, e.g. Java stack traces. | ||
|
|
||
| The `exclude_lines` sections allows to filter out lines from the sample log files. If `multiline` is used, pattern is checked after merging lines. |
There was a problem hiding this comment.
Regarding the elastic/integrations#1145:
I'm wondering if it's possible to remove lines starting with # with ingest pipeline (during processing, just skip them).
There was a problem hiding this comment.
Good idea, I will try that!
There was a problem hiding this comment.
It seems that something like #384 will be needed, otherwise errors like this happen:
ERROR: verifying test result failed: comparing test results failed: can't adjust test results: can't unmarshal event: unexpected end of JSON input
|
Not needed by now, lines can be excluded in the pipeline. |
|
Oh, I closed this thinking that it was #380 facepalm reopening, this is needed for elastic/integrations#1145. |
|
Pinging @elastic/integrations (Team:Integrations) |
|
No, this is not the PR I wanted to reopen 🤦 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Needed for mysql tests elastic/integrations#1145