Add lpkg as java package format #694

jonasagx · 2021-12-15T00:21:50Z

also cleaned up linter warning

Closes: #682

Signed-off-by: Jonas Galvão Xavier [email protected]

luhring

Thanks for doing this! One question for you to chat about re: testing

luhring · 2021-12-15T21:09:08Z

syft/pkg/cataloger/java/archive_parser.go

+	"**/*.lpkg", // Zip-compressed package used to deploy applications
+	// (aka plugins) to Liferay users that can be installed on their
+	// websites, contains .JAR files and a .PROPERTIES file, the latter
+	// has information about the application and installation requirements.


What do you think it would look like to add a test for this?

I added a file test. Their docs and tools are pretty outdated and I can't find sample projects, even on GitHub.
EDIT: found projects but building them into lpkg's is done via Liferay proprietary tools that don't have a place in our CI

linter clean up Signed-off-by: Jonas Galvão Xavier <[email protected]>

Signed-off-by: Jonas Galvão Xavier <[email protected]>

kzantow

It looks like all of these changes are gofmt or goimports except for archive_filename.go, archive_filename_test.go, and archive_parser.go, unless they are causing failures with the PR, maybe just include the relevant changes to make a review a bit more concise? I don't see any .lpkg files included, though, which makes me think there is limited testing for this - is it sufficient?

syft/pkg/apk_metadata_test.go

kzantow · 2021-12-20T20:02:15Z

syft/pkg/catalog.go

+	"github.com/anchore/syft/syft/artifact"
+	"github.com/jinzhu/copier"


nit: prefer not to include import-only changes, I think?

Signed-off-by: Jonas Galvão Xavier <[email protected]>

jonasagx · 2021-12-22T00:48:50Z

@kzantow while exploring the project my editor found places that were not properly formatted, I will try not to pollute PR too much going forward.

On testing: we don't add jar/war/hpi files within the repo, that is why there are automated building steps for most formats but war. War files are a super set of jar plus Servlet, JSP, HTML, JavaScript, but we only care about their jar content. Lpkg files are jars plus a property file. I wrote lpkg tests based off war validation that seems sufficient.

spiffcs · 2021-12-22T18:32:28Z

syft/pkg/cataloger/java/archive_parser_test.go

-						if _, ok := metadata.Manifest.Main[field]; ok {
-							delete(metadata.Manifest.Main, field)
-						}
+						delete(metadata.Manifest.Main, field)


I like that this simplifies the test! Just for my own sake, why do we no longer need this check for ok?

Go is smart:

The delete built-in function deletes the element with the specified key (m[key]) from the map. If m is nil or there is no such element, delete is a no-op.

https://pkg.go.dev/builtin#delete

spiffcs · 2021-12-22T18:34:25Z

syft/pkg/dpkg_metadata_test.go

-			var i interface{}
-			i = test.metadata
-			actual := i.(FileOwner).OwnedFiles()
+			actual := test.metadata.OwnedFiles()


Nice!

What change went in that made so that metadata no longer hast to be cast to FileOwner I don't see it in this code diff so did we update something in another repo?

Maybe this jonasagx@93d00dc

However, if test.metadata implements FileOwner there is no need to cast it. Deep equal accepts interface{} so any type can be compared.

PS: <3 hast

kzantow

This looks okay; I would definitely like to have some sort of tests beyond the one included but I understand the difficulty adding these tests due to Liferay's paid development model. The only other issue I have is that there are really only 3 files with changes but a lot of what appear to be unrelated things in the PR -- in particular goimports formatting and:

-			var i interface{}
-			i = test.metadata
-			actual := i.(FileOwner).OwnedFiles()
+			actual := test.metadata.OwnedFiles()

and

-						if _, ok := metadata.Manifest.Main[field]; ok {
-							delete(metadata.Manifest.Main, field)
-						}
+						delete(metadata.Manifest.Main, field)

I would probably just remove those changes and have the pertinent stuff in here, but they all look like improvements so I wouldn't hold up merging this on account of them!

* add lpkg support to java cataloger linter clean up Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment formatting Signed-off-by: Jonas Galvão Xavier <[email protected]> * add filename test for lpkg Signed-off-by: Jonas Galvão Xavier <[email protected]> * commment on lpkg file extension tests Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment typo Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix import format Signed-off-by: Jonas Galvão Xavier <[email protected]> * simpler test validation Signed-off-by: Jonas Galvão Xavier <[email protected]>

* add lpkg support to java cataloger linter clean up Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment formatting Signed-off-by: Jonas Galvão Xavier <[email protected]> * add filename test for lpkg Signed-off-by: Jonas Galvão Xavier <[email protected]> * commment on lpkg file extension tests Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment typo Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix import format Signed-off-by: Jonas Galvão Xavier <[email protected]> * simpler test validation Signed-off-by: Jonas Galvão Xavier <[email protected]> Signed-off-by: Christopher Angelo Phillips <[email protected]>

…hub.com/hectorj2f/syft into hectorj2f/add_dependencies_to_cyclonedx * 'hectorj2f/add_dependencies_to_cyclonedx' of https://github.com/hectorj2f/syft: (29 commits) Improve CycloneDX format output (#710) Add additional PHP metadata (#753) Update Syft formats for SyftJson (#752) Add support for "file" source type in syftjson unmarshaling (#750) remove contains file from spdx dependency generation support .sar for java ecosystem (#748) Start developer documentation (#746) Align SPDX export more with SPDX 2.2 specification (#743) Replace distro type (#742) update goreleaser with windows checksums (#740) bump stereoscope version to remove old containerd (#741) Add support for multiple output files in different formats (#732) Add support for searching for jars within archives (#734) 683 windows filepath (#735) Fix CPE encode/decode when it contains special chars (#714) support .par for java ecosystems (#727) Add arm64 support to install script (#729) Revert "bump goreleaser to v1.2 (#720)" (#731) Add a version flag (#722) Add lpkg as java package format (#694) ...

* add lpkg support to java cataloger linter clean up Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment formatting Signed-off-by: Jonas Galvão Xavier <[email protected]> * add filename test for lpkg Signed-off-by: Jonas Galvão Xavier <[email protected]> * commment on lpkg file extension tests Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment typo Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix import format Signed-off-by: Jonas Galvão Xavier <[email protected]> * simpler test validation Signed-off-by: Jonas Galvão Xavier <[email protected]> Signed-off-by: fsl <[email protected]>

* add lpkg support to java cataloger linter clean up Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment formatting Signed-off-by: Jonas Galvão Xavier <[email protected]> * add filename test for lpkg Signed-off-by: Jonas Galvão Xavier <[email protected]> * commment on lpkg file extension tests Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment typo Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix import format Signed-off-by: Jonas Galvão Xavier <[email protected]> * simpler test validation Signed-off-by: Jonas Galvão Xavier <[email protected]> Signed-off-by: Christopher Phillips <[email protected]>

* add lpkg support to java cataloger linter clean up Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment formatting Signed-off-by: Jonas Galvão Xavier <[email protected]> * add filename test for lpkg Signed-off-by: Jonas Galvão Xavier <[email protected]> * commment on lpkg file extension tests Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix comment typo Signed-off-by: Jonas Galvão Xavier <[email protected]> * fix import format Signed-off-by: Jonas Galvão Xavier <[email protected]> * simpler test validation Signed-off-by: Jonas Galvão Xavier <[email protected]>

jonasagx requested a review from a team December 15, 2021 00:23

luhring reviewed Dec 15, 2021

View reviewed changes

jonasagx self-assigned this Dec 16, 2021

jonasagx added 3 commits December 17, 2021 15:38

add lpkg support to java cataloger

54da33e

linter clean up Signed-off-by: Jonas Galvão Xavier <[email protected]>

fix comment formatting

2299591

Signed-off-by: Jonas Galvão Xavier <[email protected]>

add filename test for lpkg

511e14b

Signed-off-by: Jonas Galvão Xavier <[email protected]>

jonasagx force-pushed the main branch from d11a40c to 511e14b Compare December 17, 2021 23:38

jonasagx added 3 commits December 17, 2021 16:16

commment on lpkg file extension tests

abf33f3

Signed-off-by: Jonas Galvão Xavier <[email protected]>

fix comment typo

2ef239c

Signed-off-by: Jonas Galvão Xavier <[email protected]>

fix import format

9505caa

Signed-off-by: Jonas Galvão Xavier <[email protected]>

jonasagx requested a review from a team December 20, 2021 19:00

kzantow reviewed Dec 20, 2021

View reviewed changes

simpler test validation

eb91625

Signed-off-by: Jonas Galvão Xavier <[email protected]>

jonasagx requested a review from kzantow December 22, 2021 00:58

jonasagx mentioned this pull request Dec 22, 2021

Add version to logs #668

Closed

spiffcs reviewed Dec 22, 2021

View reviewed changes

kzantow approved these changes Dec 22, 2021

View reviewed changes

jonasagx merged commit 211b188 into anchore:main Dec 23, 2021

		"github.com/anchore/syft/syft/artifact"
		"github.com/jinzhu/copier"

Uh oh!

Add lpkg as java package format #694

Add lpkg as java package format #694

Uh oh!

Conversation

jonasagx commented Dec 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

luhring left a comment

Choose a reason for hiding this comment

Uh oh!

luhring Dec 15, 2021

Choose a reason for hiding this comment

Uh oh!

jonasagx Dec 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kzantow left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kzantow Dec 20, 2021

Choose a reason for hiding this comment

Uh oh!

jonasagx commented Dec 22, 2021

Uh oh!

spiffcs Dec 22, 2021

Choose a reason for hiding this comment

Uh oh!

jonasagx Dec 23, 2021

Choose a reason for hiding this comment

Uh oh!

spiffcs Dec 22, 2021

Choose a reason for hiding this comment

Uh oh!

jonasagx Dec 23, 2021

Choose a reason for hiding this comment

Uh oh!

kzantow left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jonasagx commented Dec 15, 2021 •

edited

Loading

jonasagx Dec 18, 2021 •

edited

Loading

kzantow left a comment •

edited

Loading