test: cleanup test_guest_os.py for multiple execution #10818

shwstppr · 2025-05-06T06:24:17Z

Description

Currently test cases in test_guest_os.py creates new guest os and mapping entries which are not always cleared. This PR makes changes to clear all entries created during tests to allow test cases to run multiple times.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Before:

[root@pr10773-t13212-kvm-ol8-marvin marvin]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=./pr10773-t13212-kvm-ol8-advanced-cfg -s -a tags=advanced --hypervisor=KVM tests/smoke/test_guest_os.py
/usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/May_06_2025_05_50_10_X8HXV0 All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_CRUD_operations_guest_OS | Status : SUCCESS ===

=== Final results are now copied to: /marvin//MarvinLogs/test_guest_os_PDFWRM ===
[root@pr10773-t13212-kvm-ol8-marvin marvin]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=./pr10773-t13212-kvm-ol8-advanced-cfg -s -a tags=advanced --hypervisor=KVM tests/smoke/test_guest_os.py
/usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/May_06_2025_06_06_52_O0ZF7T All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_CRUD_operations_guest_OS | Status : EXCEPTION ===

=== TestName: test_CRUD_operations_guest_OS_mapping | Status : EXCEPTION ===

=== TestName: test_guest_OS_mapping_check_with_hypervisor | Status : EXCEPTION ===

=== TestName: test_guest_OS_mapping_check_with_hypervisor_failure | Status : EXCEPTION ===

=== Final results are now copied to: /marvin//MarvinLogs/test_guest_os_KIXLY8 ===
[root@pr10773-t13212-kvm-ol8-marvin marvin]# cat /marvin//MarvinLogs/test_guest_os_KIXLY8/failed_plus_exceptions.txt 
2025-05-06 06:06:52,324 - CRITICAL - EXCEPTION: test_CRUD_operations_guest_OS: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 60, in testPartExecutor\n    yield\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 622, in run\n    testMethod()\n', '  File "/marvin/tests/smoke/test_guest_os.py", line 91, in test_CRUD_operations_guest_OS\n    oscategoryid=os_category.id\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 6738, in add\n    return (apiclient.addGuestOs(cmd))\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 4270, in addGuestOs\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest\n    raise e\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest\n    raise self.__lastError\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 310, in __parseAndGetResponse\n    response_cls)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/jsonHelper.py", line 155, in getResultObj\n    raise cloudstackException.CloudstackAPIException(respname, errMsg)\n', 'marvin.cloudstackException.CloudstackAPIException: Execute cmd: addguestos failed, due to: errorCode: 431, errorText:The specified Guest OS name : testCentOS already exists. Please specify a unique name\n']
2025-05-06 06:06:52,372 - CRITICAL - EXCEPTION: test_CRUD_operations_guest_OS_mapping: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 60, in testPartExecutor\n    yield\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 622, in run\n    testMethod()\n', '  File "/marvin/tests/smoke/test_guest_os.py", line 125, in test_CRUD_operations_guest_OS_mapping\n    oscategoryid=os_category.id\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 6738, in add\n    return (apiclient.addGuestOs(cmd))\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 4270, in addGuestOs\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest\n    raise e\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest\n    raise self.__lastError\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 310, in __parseAndGetResponse\n    response_cls)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/jsonHelper.py", line 155, in getResultObj\n    raise cloudstackException.CloudstackAPIException(respname, errMsg)\n', 'marvin.cloudstackException.CloudstackAPIException: Execute cmd: addguestos failed, due to: errorCode: 431, errorText:The specified Guest OS name : testCentOS already exists. Please specify a unique name\n']
2025-05-06 06:06:52,414 - CRITICAL - EXCEPTION: test_guest_OS_mapping_check_with_hypervisor: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 60, in testPartExecutor\n    yield\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 622, in run\n    testMethod()\n', '  File "/marvin/tests/smoke/test_guest_os.py", line 178, in test_guest_OS_mapping_check_with_hypervisor\n    oscategoryid=os_category.id\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 6738, in add\n    return (apiclient.addGuestOs(cmd))\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 4270, in addGuestOs\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest\n    raise e\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest\n    raise self.__lastError\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 310, in __parseAndGetResponse\n    response_cls)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/jsonHelper.py", line 155, in getResultObj\n    raise cloudstackException.CloudstackAPIException(respname, errMsg)\n', 'marvin.cloudstackException.CloudstackAPIException: Execute cmd: addguestos failed, due to: errorCode: 431, errorText:The specified Guest OS name : testOSname1 already exists. Please specify a unique name\n']
2025-05-06 06:06:52,453 - CRITICAL - EXCEPTION: test_guest_OS_mapping_check_with_hypervisor_failure: ['Traceback (most recent call last):\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 60, in testPartExecutor\n    yield\n', '  File "/usr/lib64/python3.6/unittest/case.py", line 622, in run\n    testMethod()\n', '  File "/marvin/tests/smoke/test_guest_os.py", line 237, in test_guest_OS_mapping_check_with_hypervisor_failure\n    oscategoryid=os_category.id\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 6738, in add\n    return (apiclient.addGuestOs(cmd))\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 4270, in addGuestOs\n    response = self.connection.marvinRequest(command, response_type=response, method=method)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest\n    raise e\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest\n    raise self.__lastError\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 310, in __parseAndGetResponse\n    response_cls)\n', '  File "/usr/local/lib/python3.6/site-packages/marvin/jsonHelper.py", line 155, in getResultObj\n    raise cloudstackException.CloudstackAPIException(respname, errMsg)\n', 'marvin.cloudstackException.CloudstackAPIException: Execute cmd: addguestos failed, due to: errorCode: 431, errorText:The specified Guest OS name : testOSname2 already exists. Please specify a unique name\n']

After:

[root@pr10773-t13212-kvm-ol8-marvin marvin]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=./pr10773-t13212-kvm-ol8-advanced-cfg -s -a tags=advanced --hypervisor=KVM tests/smoke/test_guest_os.py
/usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/May_06_2025_06_18_53_RN1VUC All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_CRUD_operations_guest_OS | Status : SUCCESS ===

=== Final results are now copied to: /marvin//MarvinLogs/test_guest_os_EYC44M ===
[root@pr10773-t13212-kvm-ol8-marvin marvin]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=./pr10773-t13212-kvm-ol8-advanced-cfg -s -a tags=advanced --hypervisor=KVM tests/smoke/test_guest_os.py
/usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/May_06_2025_06_19_07_JNHRTP All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_CRUD_operations_guest_OS | Status : SUCCESS ===

=== Final results are now copied to: /marvin//MarvinLogs/test_guest_os_V63BS0 ===

How did you try to break this feature and the system with this change?

Currently test cases in test_guest_os.py creat new guest os and mapping entries which are not always cleared. This PR makes changes to clear all entries created during tests to allow test cases to run multiple times. Signed-off-by: Abhishek Kumar <[email protected]>

codecov · 2025-05-06T06:32:08Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 16.13%. Comparing base (f6d0590) to head (ced4273).
Report is 1 commits behind head on 4.20.

Additional details and impacted files

@@             Coverage Diff              @@
##               4.20   #10818      +/-   ##
============================================
- Coverage     16.13%   16.13%   -0.01%     
+ Complexity    13220    13219       -1     
============================================
  Files          5651     5651              
  Lines        496740   496740              
  Branches      60183    60183              
============================================
- Hits          80148    80138      -10     
- Misses       407674   407678       +4     
- Partials       8918     8924       +6

Flag	Coverage Δ
uitests	`4.00% <ø> (ø)`
unittests	`16.98% <ø> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

DaanHoogland

clgtm

DaanHoogland · 2025-05-06T06:36:06Z

@blueorangutan package

blueorangutan · 2025-05-06T06:38:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-05-06T07:44:26Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13271

DaanHoogland · 2025-05-06T09:06:20Z

@blueorangutan test

blueorangutan · 2025-05-06T09:08:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-05-07T01:44:45Z

[SF] Trillian test result (tid-13219)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 56865 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10818-t13219-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

Pearl1594 · 2025-05-07T06:38:39Z

tools/marvin/marvin/lib/base.py

@@ -6772,10 +6773,13 @@ def list(cls, apiclient, id=None, oscategoryid=None, description=None, **kwargs)

        return (apiclient.listOsTypes(cmd))

+    def delete(self, apiclient):


out of curiosity, where is this used?

I would imagine the cleanup_resources method of cloudstackTestcase needs these.

Pearl1594

code lgtm.

boring-cyborg bot added component:integration-test component:marvin Python Warning... Python code Ahead! labels May 6, 2025

DaanHoogland approved these changes May 6, 2025

View reviewed changes

shwstppr marked this pull request as ready for review May 7, 2025 06:35

Pearl1594 reviewed May 7, 2025

View reviewed changes

Pearl1594 approved these changes May 7, 2025

View reviewed changes

DaanHoogland merged commit c1923a2 into apache:4.20 May 7, 2025
20 of 25 checks passed

DaanHoogland deleted the cleanup_test_guest_os branch May 7, 2025 06:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: cleanup test_guest_os.py for multiple execution #10818

test: cleanup test_guest_os.py for multiple execution #10818

shwstppr commented May 6, 2025 •

edited

Loading

codecov bot commented May 6, 2025 •

edited

Loading

DaanHoogland left a comment

DaanHoogland commented May 6, 2025

blueorangutan commented May 6, 2025

blueorangutan commented May 6, 2025

DaanHoogland commented May 6, 2025

blueorangutan commented May 6, 2025

blueorangutan commented May 7, 2025

Pearl1594 May 7, 2025

DaanHoogland May 7, 2025

Pearl1594 left a comment

		@@ -6772,10 +6773,13 @@ def list(cls, apiclient, id=None, oscategoryid=None, description=None, **kwargs)

		return (apiclient.listOsTypes(cmd))

		def delete(self, apiclient):

test: cleanup test_guest_os.py for multiple execution #10818

test: cleanup test_guest_os.py for multiple execution #10818

Conversation

shwstppr commented May 6, 2025 • edited Loading

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov bot commented May 6, 2025 • edited Loading

Codecov Report

DaanHoogland left a comment

Choose a reason for hiding this comment

DaanHoogland commented May 6, 2025

blueorangutan commented May 6, 2025

blueorangutan commented May 6, 2025

DaanHoogland commented May 6, 2025

blueorangutan commented May 6, 2025

blueorangutan commented May 7, 2025

Pearl1594 May 7, 2025

Choose a reason for hiding this comment

DaanHoogland May 7, 2025

Choose a reason for hiding this comment

Pearl1594 left a comment

Choose a reason for hiding this comment

shwstppr commented May 6, 2025 •

edited

Loading

codecov bot commented May 6, 2025 •

edited

Loading