Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Oct 1, 2024

Proposed commit message

sdd_all: use triple-brace templating

The mustache templating system used by ingest pipelines has two levels of
escaping available, not escaped (triple stache) and HTML escaped
(double stache) — see man mustache[1] under "tag types: variables". This can
lead to data corruption, particularly in cases where an operating system has
chosen to use a character requiring escaping in its path syntax.

[1]http://mustache.github.io/mustache.5.html

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/sec-deployment-and-devices")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' ": ('\{\{[^{][ .a-zA-Z0-9_]*[^}]}}'|\"\{\{[^{][ .a-zA-Z0-9_]*[^}]}}\")" packages/$p
		done
	)|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	sed -i -r "s/: (['\"])\{\{([^{][ .a-zA-Z0-9_]*[^}])}}['\"]/: \1{{{\2}}}\1/g" $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link https://github.com/elastic/integrations/pull/11286
	)>/dev/null 2>&1
done

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 added Integration:All Applies to all integrations [Integration not found in source] bugfix Pull request that fixes a bug issue Team:Security-Deployment and Devices DEPRECATED Deployment and Devices Security team [elastic/sec-deployment-and-devices] labels Oct 1, 2024
@efd6 efd6 self-assigned this Oct 1, 2024
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@andrewkroh andrewkroh removed the Integration:All Applies to all integrations [Integration not found in source] label Oct 1, 2024
@efd6 efd6 marked this pull request as ready for review October 1, 2024 10:15
@efd6 efd6 requested a review from a team as a code owner October 1, 2024 10:15
@elasticmachine
Copy link

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

The mustache templating system used by ingest pipelines has two levels of
escaping available, not escaped (triple stache) and HTML escaped
(double stache) — see man mustache[1] under "tag types: variables". This can
lead to data corruption, particularly in cases where an operating system has
chosen to use a character requiring escaping in its path syntax.

[1]http://mustache.github.io/mustache.5.html

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/sec-deployment-and-devices")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' ": ('\{\{[^{][ .a-zA-Z0-9_]*[^}]}}'|\"\{\{[^{][ .a-zA-Z0-9_]*[^}]}}\")" packages/$p
		done
	)|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	sed -i -r "s/: (['\"])\{\{([^{][ .a-zA-Z0-9_]*[^}])}}['\"]/: \1{{{\2}}}\1/g" $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link elastic#11286
	)>/dev/null 2>&1
done
@elasticmachine
Copy link

💚 Build Succeeded

History

  • 💚 Build #16648 succeeded 434968daf8e76ed3cec2b74980162396ced41f17

cc @efd6

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
76.8% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@andrewkroh andrewkroh added the Integration:All Applies to all integrations [Integration not found in source] label Oct 2, 2024
@andrewkroh andrewkroh added Integration:radware Radware DefensePro Logs (Deprecated) Integration:juniper_netscreen Juniper NetScreen (Deprecated) Integration:osquery Osquery Logs Integration:fortinet_forticlient Fortinet FortiClient Logs (Deprecated) Integration:juniper_junos Juniper JunOS (Deprecated) Integration:cisco_ise Cisco ISE Integration:suricata Suricata Integration:netflow NetFlow Records Integration:sophos Sophos Integration:citrix_waf Citrix Web App Firewall Integration:fortinet_fortiedr Fortinet FortiEDR Logs and removed Integration:All Applies to all integrations [Integration not found in source] labels Oct 2, 2024
@andrewkroh andrewkroh added Integration:juniper_srx Juniper SRX Integration:pfsense pfSense (Community supported) Integration:snort Snort Integration:bluecoat Blue Coat Director Logs (Deprecated) Integration:fortinet_fortigate Fortinet FortiGate Firewall Logs labels Oct 2, 2024
@efd6 efd6 merged commit 992a01e into elastic:main Oct 2, 2024
4 of 5 checks passed
@elastic-vault-github-plugin-prod

Package bluecoat - 0.17.3 containing this change is available at https://epr.elastic.co/search?package=bluecoat

@elastic-vault-github-plugin-prod

Package cef - 2.17.3 containing this change is available at https://epr.elastic.co/search?package=cef

@elastic-vault-github-plugin-prod

Package checkpoint - 1.34.1 containing this change is available at https://epr.elastic.co/search?package=checkpoint

@elastic-vault-github-plugin-prod

Package cisco_ise - 1.23.1 containing this change is available at https://epr.elastic.co/search?package=cisco_ise

@elastic-vault-github-plugin-prod

Package citrix_waf - 1.16.1 containing this change is available at https://epr.elastic.co/search?package=citrix_waf

@elastic-vault-github-plugin-prod

Package fortinet_forticlient - 1.10.3 containing this change is available at https://epr.elastic.co/search?package=fortinet_forticlient

@elastic-vault-github-plugin-prod

Package fortinet_fortiedr - 1.15.1 containing this change is available at https://epr.elastic.co/search?package=fortinet_fortiedr

@elastic-vault-github-plugin-prod

Package fortinet_fortigate - 1.25.8 containing this change is available at https://epr.elastic.co/search?package=fortinet_fortigate

@elastic-vault-github-plugin-prod

Package juniper_junos - 0.10.3 containing this change is available at https://epr.elastic.co/search?package=juniper_junos

@elastic-vault-github-plugin-prod

Package juniper_netscreen - 0.10.3 containing this change is available at https://epr.elastic.co/search?package=juniper_netscreen

@elastic-vault-github-plugin-prod

Package juniper_srx - 1.21.1 containing this change is available at https://epr.elastic.co/search?package=juniper_srx

@elastic-vault-github-plugin-prod

Package modsecurity - 1.18.1 containing this change is available at https://epr.elastic.co/search?package=modsecurity

@elastic-vault-github-plugin-prod

Package netflow - 2.19.1 containing this change is available at https://epr.elastic.co/search?package=netflow

@elastic-vault-github-plugin-prod

Package netscout - 0.20.1 containing this change is available at https://epr.elastic.co/search?package=netscout

@elastic-vault-github-plugin-prod

Package osquery - 1.19.1 containing this change is available at https://epr.elastic.co/search?package=osquery

@elastic-vault-github-plugin-prod

Package pfsense - 1.20.1 containing this change is available at https://epr.elastic.co/search?package=pfsense

@elastic-vault-github-plugin-prod

Package qnap_nas - 1.20.1 containing this change is available at https://epr.elastic.co/search?package=qnap_nas

@elastic-vault-github-plugin-prod

Package radware - 0.19.1 containing this change is available at https://epr.elastic.co/search?package=radware

@elastic-vault-github-plugin-prod

Package snort - 1.15.1 containing this change is available at https://epr.elastic.co/search?package=snort

@elastic-vault-github-plugin-prod

Package sophos - 3.9.1 containing this change is available at https://epr.elastic.co/search?package=sophos

@elastic-vault-github-plugin-prod

Package suricata - 2.21.3 containing this change is available at https://epr.elastic.co/search?package=suricata

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
The mustache templating system used by ingest pipelines has two levels of
escaping available, not escaped (triple stache) and HTML escaped
(double stache) — see man mustache[1] under "tag types: variables". This can
lead to data corruption, particularly in cases where an operating system has
chosen to use a character requiring escaping in its path syntax.

[1]http://mustache.github.io/mustache.5.html

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/sec-deployment-and-devices")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' ": ('\{\{[^{][ .a-zA-Z0-9_]*[^}]}}'|\"\{\{[^{][ .a-zA-Z0-9_]*[^}]}}\")" packages/$p
		done
	)|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	sed -i -r "s/: (['\"])\{\{([^{][ .a-zA-Z0-9_]*[^}])}}['\"]/: \1{{{\2}}}\1/g" $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link elastic#11286
	)>/dev/null 2>&1
done
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
The mustache templating system used by ingest pipelines has two levels of
escaping available, not escaped (triple stache) and HTML escaped
(double stache) — see man mustache[1] under "tag types: variables". This can
lead to data corruption, particularly in cases where an operating system has
chosen to use a character requiring escaping in its path syntax.

[1]http://mustache.github.io/mustache.5.html

[git-generate]
for f in $(
	(
		for p in $(
			yq 'select(.owner.github == "elastic/sec-deployment-and-devices")|.name' packages/**/manifest.yml \
			| grep -v -- '---'
		); do
			rg -l -g '*.yml' ": ('\{\{[^{][ .a-zA-Z0-9_]*[^}]}}'|\"\{\{[^{][ .a-zA-Z0-9_]*[^}]}}\")" packages/$p
		done
	)|grep "elasticsearch/ingest_pipeline"|sort|uniq
); do
	sed -i -r "s/: (['\"])\{\{([^{][ .a-zA-Z0-9_]*[^}])}}['\"]/: \1{{{\2}}}\1/g" $f
done
for p in $(git diff --name-only HEAD~1|cut -d/ -f1,2|sort|uniq); do
	(
		cd $p
		elastic-package test pipeline -g
		elastic-package changelog add \
			--description "Use triple-brace Mustache templating when referencing variables in ingest pipelines." \
			--type bugfix \
			--next patch \
			--link elastic#11286
	)>/dev/null 2>&1
done
@efd6 efd6 deleted the 7641-sdd_all branch February 5, 2025 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:bluecoat Blue Coat Director Logs (Deprecated) Integration:cef Common Event Format (CEF) Integration:checkpoint Check Point Integration:cisco_ise Cisco ISE Integration:citrix_waf Citrix Web App Firewall Integration:fortinet_forticlient Fortinet FortiClient Logs (Deprecated) Integration:fortinet_fortiedr Fortinet FortiEDR Logs Integration:fortinet_fortigate Fortinet FortiGate Firewall Logs Integration:juniper_junos Juniper JunOS (Deprecated) Integration:juniper_netscreen Juniper NetScreen (Deprecated) Integration:juniper_srx Juniper SRX Integration:modsecurity ModSecurity Audit (Community supported) Integration:netflow NetFlow Records Integration:netscout Arbor Peakflow SP Logs (Deprecated) Integration:osquery Osquery Logs Integration:pfsense pfSense (Community supported) Integration:qnap_nas QNAP NAS (Community supported) Integration:radware Radware DefensePro Logs (Deprecated) Integration:snort Snort Integration:sophos Sophos Integration:suricata Suricata Team:Security-Deployment and Devices DEPRECATED Deployment and Devices Security team [elastic/sec-deployment-and-devices]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants