Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Support XenServer 8.4 / XCP 8.3 - make scripts python3 compatible #10684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 33 commits into
base: 4.20
Choose a base branch
from

Conversation

Pearl1594
Copy link
Contributor

@Pearl1594 Pearl1594 commented Apr 9, 2025

Description

This PR is just to test #10483

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

rohityadavcloud and others added 6 commits February 28, 2025 11:07
This aims to add support for XenServer 8.4 and XCP-ng 8.3.

Signed-off-by: Rohit Yadav <[email protected]>
Signed-off-by: Rohit Yadav <[email protected]>
Signed-off-by: Rohit Yadav <[email protected]>
Signed-off-by: Rohit Yadav <[email protected]>
@boring-cyborg boring-cyborg bot added component:XenServer Python Warning... Python code Ahead! labels Apr 9, 2025
@Pearl1594
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link

codecov bot commented Apr 9, 2025

Codecov Report

Attention: Patch coverage is 0% with 203 lines in your changes missing coverage. Please review.

Project coverage is 16.13%. Comparing base (f6d0590) to head (1ae4276).

Files with missing lines Patch % Lines
.../main/java/com/xensource/xenapi/ConnectionNew.java 0.00% 156 Missing ⚠️
...rce/wrapper/xenbase/CitrixStartCommandWrapper.java 0.00% 18 Missing ⚠️
...cloudstack/storage/template/UploadManagerImpl.java 0.00% 10 Missing ⚠️
...isor/xenserver/discoverer/XcpServerDiscoverer.java 0.00% 4 Missing ⚠️
...ervisor/xenserver/resource/CitrixResourceBase.java 0.00% 4 Missing ⚠️
...rvisor/xenserver/resource/XcpServer83Resource.java 0.00% 4 Missing ⚠️
...rvisor/xenserver/resource/Xenserver84Resource.java 0.00% 4 Missing ⚠️
...or/xenserver/resource/XenServerConnectionPool.java 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               4.20   #10684      +/-   ##
============================================
- Coverage     16.13%   16.13%   -0.01%     
- Complexity    13220    13221       +1     
============================================
  Files          5651     5654       +3     
  Lines        496740   496937     +197     
  Branches      60183    60232      +49     
============================================
+ Hits          80148    80160      +12     
- Misses       407674   407853     +179     
- Partials       8918     8924       +6     
Flag Coverage Δ
uitests 4.00% <ø> (ø)
unittests 16.98% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12999

@weizhouapache
Copy link
Member

@blueorangutan test ol8 xenserver-84 keepEnv

@blueorangutan
Copy link

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + xenserver-84) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian Build Failed (tid-12931)

Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lint issues

@rohityadavcloud
Copy link
Member

Good to see this, @Pearl1594 can we also add support for XCP-ng 8.3 (& related files as needed)? cc @weizhouapache @harikrishna-patnala

@Pearl1594
Copy link
Contributor Author

yes @rohityadavcloud . Working with @harikrishna-patnala towards that.

@Pearl1594
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link
Member

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

I have nothing to advise.
good job @Pearl1594

@blueorangutan
Copy link

[SF] Trillian test result (tid-13179)
Environment: xenserver-84 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 73095 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13179-xenserver-84.zip
Smoke tests completed. 135 look OK, 6 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_07_deploy_vm_with_extraconfig_xenserver Error 121.41 test_deploy_vm_extra_config_data.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 9.82 test_network.py
test_02_create_volume Error 3.28 test_resource_names.py
test_04_extract_template Failure 1.10 test_templates.py
test_02_restore_vm_strict_tags_failure Failure 90.25 test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure Failure 77.22 test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure Failure 6.76 test_vm_strict_host_tags.py
test_06_download_detached_volume Failure 198.84 test_volumes.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-13180)
Environment: xcpng83 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 77774 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13180-xcpng83.zip
Smoke tests completed. 135 look OK, 6 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_07_deploy_vm_with_extraconfig_xenserver Error 133.70 test_deploy_vm_extra_config_data.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 10.76 test_network.py
test_02_create_volume Error 2.26 test_resource_names.py
test_04_extract_template Failure 1.11 test_templates.py
test_02_restore_vm_strict_tags_failure Failure 78.94 test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure Failure 81.12 test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure Failure 4.74 test_vm_strict_host_tags.py
test_06_download_detached_volume Failure 204.97 test_volumes.py

@Pearl1594
Copy link
Contributor Author

3509d13 fixes the issue with the way extra config is passed.
Before fix:

[root@ref-trl-8431-x-mol8-pearl-dsilva-marvin ~]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=/marvin/ref-trl-8431-x-Mol8-pearl-dsilva-advanced-cfg -s -a tags=before --hypervisor=Xenserver /marvin/tests/smoke/test_deploy_vm_extra_config_data.py
/usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/Apr_30_2025_09_14_32_YGPKI3 All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_07_deploy_vm_with_extraconfig_xenserver | Status : EXCEPTION ===

=== Final results are now copied to: /marvin//MarvinLogs/test_deploy_vm_extra_config_data_L96KRM ===

Error thrown on Xen:

2025-04-30 08:50:18,876 DEBUG [c.c.a.ApiServlet] (qtp1390913202-17:[ctx-0661945c, ctx-8c964097, ctx-d0696d21]) (logid:f3928490) ===END===  10.0.112.109 -- GET  jobid=73e90964-c8a9-4dd2-b9d5-1afb8c54c037&command=queryAsyncJobResult&response=json&apiKey=LIN6rqXuaJwMPfGYFh13qDwYz5VNNz1J2J6qIOWcd3oLQOq0WtD4CwRundBL6rzXToa3lQOC_vKjI3nkHtiD8Q&signature=zhix3C3GW%2BmFsjrPyyNggORyUas%3D
2025-04-30 08:50:19,166 WARN  [c.c.h.x.r.XcpServer83Resource] (DirectAgent-244:[ctx-e32986ce]) (logid:73e90964) Task failed! Task record:                 uuid: 6e081024-12f1-7b91-8535-c7bd9d2a6006
           nameLabel: Async.VM.start_on
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Wed Apr 30 08:52:59 UTC 2025
            finished: Wed Apr 30 08:53:05 UTC 2025
              status: failure
          residentOn: com.xensource.xenapi.Host@aa444b17
            progress: 1.0
                type: <none/>
              result:
           errorInfo: [INTERNAL_ERROR, xenopsd internal error: Domain.Domain_build_pre_failed("Calling 'shadow_allocation_set 3 MiB' failed: Xenctrl.Error(\"22: Invalid argument\")")]
         otherConfig: {debug_info:cancel_points_seen=1}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []
           backtrace: (((process xapi)(filename ocaml/xapi-client/client.ml)(line 7))((process xapi)(filename ocaml/xapi-client/client.ml)(line 19))((process xapi)(filename ocaml/xapi-client/client.ml)(line 5845))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 146))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 1938))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 1922))((process xapi)(filename ocaml/xapi/rbac.ml)(line 191))((process xapi)(filename ocaml/xapi/rbac.ml)(line 200))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 75)))

2025-04-30 08:50:19,183 INFO  [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-84fdefd7]) (logid:fc0a7c8f) No inactive management server node found
2025-04-30 08:50:19,183 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-84fdefd7]) (logid:fc0a7c8f) Peer scan is finished. profiler: Done. Duration: 2ms , profilerQueryActiveList: Done. Duration: 1ms, , profilerSyncClusterInfo: Done. Duration: 0ms, profilerInvalidatedNodeList: Done. Duration: 0ms, profilerRemovedList: Done. Duration: 0ms

Post fix:

[root@ref-trl-8431-x-mol8-pearl-dsilva-marvin ~]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=/marvin/ref-trl-8431-x-Mol8-pearl-dsilva-advanced-cfg -s -a tags=after --hypervisor=Xenserver /marvin/tests/smoke/test_deploy_vm_extra_config_data.py
/usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/Apr_30_2025_09_16_42_MFF43T All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
====Trying SSH Connection: Host:10.0.35.194 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.0.35.194 port : 22 SUCCESSFUL===
{Cmd: xe vm-list name-label=i-2-9-VM params=uuid  via Host: 10.0.35.194} {returns: ['uuid ( RO)    : 7e391f0c-1fd5-329a-660e-0c42f196eb5e', '', '']}
{Cmd: xe vm-param-get param-name=HVM-boot-policy uuid=7e391f0c-1fd5-329a-660e-0c42f196eb5e via Host: 10.0.35.194} {returns: ['""']}
{Cmd: xe vm-param-get param-name=PV-bootloader uuid=7e391f0c-1fd5-329a-660e-0c42f196eb5e via Host: 10.0.35.194} {returns: ['pygrub']}
{Cmd: xe vm-param-get param-name=PV-args uuid=7e391f0c-1fd5-329a-660e-0c42f196eb5e via Host: 10.0.35.194} {returns: ['hvc0']}
=== TestName: test_07_deploy_vm_with_extraconfig_xenserver | Status : SUCCESS ===

=== Final results are now copied to: /marvin//MarvinLogs/test_deploy_vm_extra_config_data_Y8S84E ===

@Pearl1594
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13232

@Pearl1594
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link

sonarqubecloud bot commented May 5, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 40%)
C Maintainability Rating on New Code (required ≥ B)
C Reliability Rating on New Code (required ≥ B)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13263

@Pearl1594
Copy link
Contributor Author

@blueorangutan test matrix

@blueorangutan
Copy link

@Pearl1594 a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests

@Pearl1594
Copy link
Contributor Author

@blueorangutan test ol8 xenserver-84

@blueorangutan
Copy link

@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + xenserver-84) has been kicked to run smoke tests

@Pearl1594
Copy link
Contributor Author

@blueorangutan test ol8 xcpng83

@blueorangutan
Copy link

@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + xcpng83) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-13206)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 56594 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13206-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

@blueorangutan
Copy link

[SF] Trillian test result (tid-13207)
Environment: kvm-ubuntu22 (x2), Advanced Networking with Mgmt server u22
Total time taken: 61798 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13207-kvm-ubuntu22.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_oobm_multiple_mgmt_server_ownership Failure 30.88 test_outofbandmanagement.py
test_02_list_cpvm_vm Failure 0.04 test_ssvm.py
test_04_cpvm_internals Failure 0.05 test_ssvm.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-13208)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 67120 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13208-vmware-70u3.zip
Smoke tests completed. 138 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_deployVMInSharedNetwork Error 181.12 test_network.py
test_02_restore_vm_with_disk_offering Error 63.66 test_restore_vm.py
test_03_restore_vm_with_disk_offering_custom_size Error 59.47 test_restore_vm.py
test_02_restore_vm_strict_tags_failure Error 55.85 test_vm_strict_host_tags.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-13210)
Environment: xenserver-84 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 67815 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13210-xenserver-84.zip
Smoke tests completed. 140 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 7.50 test_network.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-13211)
Environment: xcpng83 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 74220 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13211-xcpng83.zip
Smoke tests completed. 140 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 7.55 test_network.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-13209)
Environment: xcpng82 (x2), Advanced Networking with Mgmt server ol9
Total time taken: 82436 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10684-t13209-xcpng82.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 10.60 test_network.py
test_10_list_volumes Failure 371.05 test_volumes.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

8 participants