Acceptances tests not indicating failures #156
Description
Running make acceptance-linux results in:
+ mkdir -p ./.tmp/results
+ cp /tmp/syft-acceptance-test-rpm-HGwqJC/acceptance-rpm-centos-8.2.2004.json ./.tmp/results
+ ./test/acceptance/compare.py ./test/acceptance/test-fixtures/acceptance-centos-8.2.2004.json /tmp/syft-acceptance-test-rpm-HGwqJC/acceptance-rpm-centos-8.2.2004.json
+ tee ./.tmp/results/acceptance-rpm.txt
Traceback (most recent call last):
Comparing two Syft reports...
File "./test/acceptance/compare.py", line 108, in <module>
rc = main(sys.argv[1], sys.argv[2])
File "./test/acceptance/compare.py", line 41, in main
report2_packages, report2_metadata = report2_obj.packages()
File "./test/acceptance/compare.py", line 31, in packages
metadata=repr(entry.get("metadata", "")), sources=repr(entry["sources"])
KeyError: 'sources'
+ cleanup
+ rm -rf /tmp/syft-acceptance-test-rpm-HGwqJC
However, the exit code is 0 (but should be 1). Two fixes are needed:
- the compare script should be updated to support the new JSON format
- the make target should fail on bad comparisons