Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 4127856

Browse files
MichaelKatsoulisMichaelKatsoulis
committed
Merge remote-tracking branch 'upstream/main' into add-prefilters-k8s-dashboards
2 parents 95f4a1d + 8da35ed commit 4127856

File tree

2,239 files changed

+31031
-468158
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

2,239 files changed

+31031
-468158
lines changed

‎.github/CODEOWNERS‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@
7474
/packages/fim @elastic/security-external-integrations
7575
/packages/fireeye @elastic/security-external-integrations
7676
/packages/fleet_server @elastic/elastic-agent-control-plane
77+
/packages/forgerock @elastic/security-external-integrations
7778
/packages/fortinet @elastic/security-external-integrations
7879
/packages/fortinet_forticlient @elastic/security-external-integrations
7980
/packages/fortinet_fortiedr @elastic/security-external-integrations
@@ -188,7 +189,7 @@
188189
/packages/ti_recordedfuture @elastic/security-external-integrations
189190
/packages/ti_threatq @elastic/security-external-integrations
190191
/packages/ti_util @elastic/security-external-integrations
191-
/packages/tomcat @elastic/security-external-integrations
192+
/packages/tomcat @elastic/obs-service-integrations
192193
/packages/traefik @elastic/obs-service-integrations
193194
/packages/trend_micro_vision_one @elastic/security-external-integrations
194195
/packages/udp @elastic/security-external-integrations
@@ -211,3 +212,4 @@
211212
/packages/lmd @elastic/ml-ui
212213
/packages/gcp_metrics @elastic/obs-cloud-monitoring
213214
/packages/airflow @elastic/obs-service-integrations
215+
/packages/statsd_input @elastic/obs-service-integrations

‎docs/dashboard_guidelines.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Dashboard guidelines
2+
3+
A [Kibana dashboard][1] is a set of one or more panels, also referred as visualizations. Panels display data in charts, tables, maps, and more. Dashboards support several types of panels to display your data, and several options to create panels.
4+
5+
The goal of each integration dashboard is to:
6+
7+
* Provide a way to explore ingested data out of the box.
8+
* Provide an overview of the monitored resources through installing the integration.
9+
10+
Each integration package should contain one or more dashboards.
11+
12+
## Out of date fields in dashboards
13+
14+
The dashboards must be updated to reflect any changes to field names or types. If a PR updates a field name or type, make sure it is correctly updated in any dashboard the field is being used into.
15+
16+
## TSDB visualizations
17+
18+
Migrate the dashboards from TSVB to Lens where possible. If it's not possible, please engage with the Kibana team to identify any gaps that prevent from full TSVB to Lens dashboard migration.
19+
20+
## Visualizations by value, not by reference
21+
22+
Kibana visualizations can be added in a dashboard by value or by reference. Historically by value did not exist. Switching to value has the advantage that the dashboards are fully self contained and only need a single request to be installed.
23+
24+
To achieve this:
25+
- Migrate existing dashboards from by reference to by value.
26+
- Create new dashboards adding visualizations by value.
27+
28+
A migration script has been created to help with the migration: [flash1293/legacy_vis_analyzer][2]
29+
30+
## Visualizations should contain a filter
31+
32+
Kibana visualizations can define a filter to avoid performance issues querying all `metrics-*` or `logs-*` indices.
33+
34+
It is recommended to set a filter in each visualization at least by the required `data_stream.dataset`. More details about the Elastic data stream naming scheme [here][4].
35+
36+
Example of this filter:
37+
- [System visualization - User Logon Dashboard][3]
38+
39+
40+
[1]: https://www.elastic.co/guide/en/kibana/current/dashboard.html
41+
[2]: https://github.com/flash1293/legacy_vis_analyzer
42+
[3]: https://github.com/elastic/integrations/blob/5176089e30cf2932d6e5ca7c90caa2ab9a237bee/packages/system/kibana/visualization/system-18348f30-a24d-11e9-a422-d144027429da.json#L9
43+
[4]: https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme

‎docs/generic_guidelines.md‎

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -99,10 +99,7 @@ For more details, see the [Documentation guidelines](./documentation_guidelines.
9999
#### Updated integration content
100100

101101
Integration packages should provide out-of-the-box dashboards.
102-
103-
- The dashboards should be updated to reflect any changes to metric names or types. (A typical issue with our dashboards is that they show raw monotonically increasing counters instead of gauges).
104-
- Some packages may require new dashboards.
105-
- Migrate the dashboards from TSVB to Lens where possible. (Kibana also requested to identify any gaps that prevent from full TSVB to Lens dashboard migration for their prioritization).
102+
For more details, see the [Dashboard guidelines](./dashboard_guidelines.md).
106103

107104
#### Content for elastic.co/integrations
108105

@@ -116,3 +113,23 @@ Each integration will be listed on the public website elastic.co/integrations an
116113
It's advised to set integration policies in the Fleet. Every integration and agent should be visible in Fleet and users should be able to add the integration directly from the integration list. This will lead to better cohesion since it will provide a consistent experience across integrations, allow users to add several integrations at once, and avoid sending them back and forth between multiple apps. It will also allow users to discover new integrations in the list.
117114

118115
Elastic products will also have the option to provide a curated UI for settings that are difficult to put in Fleet. It's up to the product to decide how much flexibility they want to provide in changing the configuration directly from Fleet. This will depend on the use case and if it makes sense. Some level of configuration is recommended though.
116+
117+
#### Asset tagging and metadata
118+
119+
When assets are installed through Fleet, some metadata will be added by default.
120+
121+
For Elasticsearch assets like Index Templates and Ingest Pipelines, a `_meta` property will be added to the asset as follows
122+
123+
```json
124+
{
125+
"managed_by": "fleet",
126+
"managed": true,
127+
"package": {
128+
"name": "<package name>"
129+
}
130+
}
131+
```
132+
133+
For Kibana assets, [tags](https://www.elastic.co/guide/en/kibana/current/managing-tags.html) will be generated in addition to the `_meta` property:
134+
- One tag with a `name` matching the package's `title` property
135+
- The `Managed` tag, which Kibana uses to recognize "system" assets, or those that are installed by Kibana itself instead of generated by an end user

‎docs/tips_for_building_integrations.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,15 @@ $ ./elastic-package help
102102

103103
Good candidates: *Filebeat running on AWS EC2 machine*
104104

105+
5. If package relies on some feature or a field, available only in a specific stack or beats version, `kibana.version` condition should be adjusted accordingly in the package's `manifest.yml`:
106+
```yaml
107+
conditions:
108+
kibana.version: '^8.7.0'
109+
```
110+
> Note: The package version with such condition as above will be only available in Kibana version >=8.7.0
111+
112+
> Note: Changing dashboards and visualizations using an unreleased version of Kibana might be unsafe since the Kibana Team might make changes to the Kibana code and potentially the data models. There is no guarantee that your changes won't be broken by the time new Kibana version is released.
113+
105114
#### CI
106115

107116
1. Run `elastic-package check` and `elastic-package test` locally.

‎packages/1password/_dev/build/build.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
dependencies:
22
ecs:
3-
reference: git@v8.5.1
3+
reference: git@8.6

‎packages/1password/changelog.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# newer versions go on top
2+
- version: "1.8.0"
3+
changes:
4+
- description: Update package to ECS 8.6.0.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/4576
27
- version: "1.7.1"
38
changes:
49
- description: Migrate the visualizations to by value in dashboards to minimize the saved object clutter and reduce time to load

‎packages/1password/data_stream/item_usages/_dev/test/pipeline/test-itemusages.json-expected.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
{
44
"@timestamp": "2021-08-30T18:57:42.484Z",
55
"ecs": {
6-
"version": "8.5.0"
6+
"version": "8.6.0"
77
},
88
"event": {
99
"action": "reveal",
@@ -76,7 +76,7 @@
7676
{
7777
"@timestamp": "2021-08-30T19:10:00.123Z",
7878
"ecs": {
79-
"version": "8.5.0"
79+
"version": "8.6.0"
8080
},
8181
"event": {
8282
"category": [

‎packages/1password/data_stream/item_usages/elasticsearch/ingest_pipeline/default.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ processors:
1616
#######################
1717
- set:
1818
field: ecs.version
19-
value: '8.5.0'
19+
value: '8.6.0'
2020
# Sets event.created from the @timestamp field generated by filebeat before being overwritten further down
2121
- set:
2222
field: event.created

‎packages/1password/data_stream/item_usages/sample_event.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
"type": "logs"
1414
},
1515
"ecs": {
16-
"version": "8.5.0"
16+
"version": "8.6.0"
1717
},
1818
"elastic_agent": {
1919
"id": "8652330e-4de6-4596-a16f-4463a6c56e9e",

‎packages/1password/data_stream/signin_attempts/_dev/test/pipeline/test-signinattempts.json-expected.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
{
44
"@timestamp": "2021-08-11T14:28:03.000Z",
55
"ecs": {
6-
"version": "8.5.0"
6+
"version": "8.6.0"
77
},
88
"event": {
99
"action": "success",
@@ -78,7 +78,7 @@
7878
{
7979
"@timestamp": "2021-08-11T15:04:22.000Z",
8080
"ecs": {
81-
"version": "8.5.0"
81+
"version": "8.6.0"
8282
},
8383
"event": {
8484
"action": "credentials_failed",

0 commit comments

Comments
 (0)