Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add support for network data in Config Drive #9329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Aug 26, 2024
Merged

Add support for network data in Config Drive #9329

merged 7 commits into from
Aug 26, 2024

Conversation

vishesh92
Copy link
Member

@vishesh92 vishesh92 commented Jul 3, 2024 •
edited
Loading

Description

This PR fixes #2872
PR adds support for Dhcp & Dns capabilities for ConfigDrive network element.

Docs PR: apache/cloudstack-documentation#412

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

  1. Create a network offering with DHCP, DNS & UserData provider with ConfigDrive
  2. Create a network with the above offering
  3. Launch an instance using a cloud init configured template.
  4. Instance gets the assigned IP, DNS & Userdata.

Note

While attaching secondary nics, interface is assigned an IP on boot only if the instance was started for the first time.
Updating of IPs & NICs after first boot doesn't reflect on the instance even after reboot.

How did you try to break this feature and the system with this change?

@codecov Codecov
Copy link

codecov bot commented Jul 3, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 57.27700% with 91 lines in your changes missing coverage. Please review.

Project coverage is 15.71%. Comparing base (47a6b70) to head (6c26bc4).
Report is 31 commits behind head on main.

Files Patch % Lines
...oud/network/element/ConfigDriveNetworkElement.java 21.42% 42 Missing and 2 partials ⚠️
...tack/engine/orchestration/NetworkOrchestrator.java 0.00% 21 Missing ⚠️
...dstack/storage/configdrive/ConfigDriveBuilder.java 83.33% 2 Missing and 16 partials ⚠️
.../main/java/com/cloud/network/NetworkModelImpl.java 0.00% 7 Missing ⚠️
...oudstack/storage/configdrive/ConfigDriveUtils.java 95.23% 0 Missing and 1 partial ⚠️
Additional details and impacted files 
@@             Coverage Diff              @@
##               main    #9329      +/-   ##
============================================
+ Coverage     15.53%   15.71%   +0.17%     
- Complexity    11988    12269     +281     
============================================
  Files          5496     5502       +6     
  Lines        481450   490878    +9428     
  Branches      60738    64541    +3803     
============================================
+ Hits          74809    77156    +2347     
- Misses       398364   405194    +6830     
- Partials       8277     8528     +251
Flag Coverage Δ
uitests 4.98% <ø> (+0.80%) ⬆️
unittests 16.45% <57.27%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@vishesh92 vishesh92 force-pushed the configdrive-network-data branch from 227be3d to 9d59c87 Compare July 3, 2024 07:58
@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 10238

@blueorangutan
Copy link

[SF] Trillian Build Failed (tid-10722)

@blueorangutan
Copy link

[SF] Trillian test result (tid-10723)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t10723-kvm-centos7.zip
Smoke tests completed. 0 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
login_test_saml_user Error 3.05 test_login.py

@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el7 ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 10255

@vishesh92 vishesh92 force-pushed the configdrive-network-data branch from 8ffa213 to 6439aa7 Compare July 5, 2024 09:50
@vishesh92
Copy link
Member Author

@blueorangutan package

@vishesh92 vishesh92 force-pushed the configdrive-network-data branch 2 times, most recently from b1f0fda to b8a1a90 Compare July 5, 2024 09:51
@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10261

@vishesh92 vishesh92 changed the title WIP: Add support for network data in Config Drive Add support for network data in Config Drive Jul 9, 2024
@rohityadavcloud rohityadavcloud mentioned this pull request Jul 9, 2024
Closed
@vishesh92 vishesh92 force-pushed the configdrive-network-data branch from 73d2970 to e438a3f Compare July 9, 2024 13:17
@vishesh92 vishesh92 mentioned this pull request Jul 10, 2024
Merged
harikrishna-patnala
harikrishna-patnala reviewed Jul 10, 2024
View reviewed changes
Copy link
Contributor

@harikrishna-patnala harikrishna-patnala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few comments from my side @vishesh92 can you please address them

harikrishna-patnala
harikrishna-patnala reviewed Jul 10, 2024
View reviewed changes
Copy link
Contributor

@harikrishna-patnala harikrishna-patnala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another question, how this needs to be configured in the network offering ? like we choose the config drive as the provider for userdata while creating the network offering

@vishesh92
Copy link
Member Author

Another question, how this needs to be configured in the network offering ? like we choose the config drive as the provider for userdata while creating the network offering

Yes. You just need to select confidDrive for Dhcp & DNS while creating the network offering.

@rohityadavcloud rohityadavcloud added this to the 4.20.0.0 milestone Jul 11, 2024
@vishesh92 vishesh92 force-pushed the configdrive-network-data branch from ccf3568 to ee9507e Compare July 12, 2024 08:39
@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10681

@blueorangutan
Copy link

[SF] Trillian test result (tid-11091)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57487 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11091-kvm-ol8.zip
Smoke tests completed. 135 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_DeployVmAffinityGroup Error 1.46 test_affinity_groups.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 70.60 test_network.py
test_01_non_strict_host_anti_affinity Error 3.61 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 3.65 test_nonstrict_affinity_group.py
test_02_unsecure_vm_migration Error 425.52 test_vm_life_cycle.py

@blueorangutan
Copy link

[SF] Trillian Build Failed (tid-11103)

@blueorangutan
Copy link

[SF] Trillian test result (tid-11106)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54415 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11106-kvm-ol8.zip
Smoke tests completed. 136 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_DeployVmAffinityGroup Error 1.48 test_affinity_groups.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown Error 212.25 test_network.py
test_01_non_strict_host_anti_affinity Error 2.58 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 3.58 test_nonstrict_affinity_group.py

@vishesh92 vishesh92 force-pushed the configdrive-network-data branch from 665cdff to 425af71 Compare August 21, 2024 05:41
@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10722

@blueorangutan
Copy link

[SF] Trillian test result (tid-11124)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 53325 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11124-kvm-ol8.zip
Smoke tests completed. 135 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_DeployVmAffinityGroup Error 1.48 test_affinity_groups.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown Error 192.56 test_network.py
test_01_non_strict_host_anti_affinity Error 3.58 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 3.58 test_nonstrict_affinity_group.py
test_01_secure_vm_migration Error 135.40 test_vm_life_cycle.py
test_01_secure_vm_migration Error 135.40 test_vm_life_cycle.py

@vishesh92 vishesh92 force-pushed the configdrive-network-data branch from 425af71 to 6c26bc4 Compare August 22, 2024 12:15
@vishesh92
Copy link
Member Author

@blueorangutan package

@blueorangutan
Copy link

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10753

@DaanHoogland
Copy link
Contributor

@vishesh92 does this need more testing?

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-11145)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 58696 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11145-kvm-ol8.zip
Smoke tests completed. 137 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 2.30 test_network.py
test_01_secure_vm_migration Error 404.46 test_vm_life_cycle.py
test_02_unsecure_vm_migration Error 450.63 test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration Error 359.84 test_vm_life_cycle.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-11152)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 50517 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9329-t11152-kvm-ol8.zip
Smoke tests completed. 139 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

@DaanHoogland
Copy link
Contributor

@vishesh92 , this had quite some changes since functional tests, does it need a re-visit, or can we merge now?

@vishesh92
Copy link
Member Author

@vishesh92 , this had quite some changes since functional tests, does it need a re-visit, or can we merge now?

We can merge now

@DaanHoogland DaanHoogland merged commit bc28665 into main Aug 26, 2024
43 of 48 checks passed
@DaanHoogland DaanHoogland deleted the configdrive-network-data branch August 26, 2024 08:54
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Aug 30, 2024
@vishesh92 @dhslove
Add support for network data in Config Drive (apache#9329)
7f26e6d
@weizhouapache
Copy link
Member

@vishesh92
there are lots of smoke test failures with test_01_deployVMInSharedNetwork, mostly happened on vmware environments.

worth checking , maybe related to the template ?

@vishesh92
Copy link
Member Author

@vishesh92 there are lots of smoke test failures with test_01_deployVMInSharedNetwork, mostly happened on vmware environments.

worth checking , maybe related to the template ?

@weizhouapache Let me check and create a fix pr for that.

@kiranchavala kiranchavala mentioned this pull request Sep 19, 2024
Closed
@phsm phsm mentioned this pull request Dec 12, 2024
Open
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@harikrishna-patnala harikrishna-patnala harikrishna-patnala approved these changes

@vladimirpetrov vladimirpetrov vladimirpetrov approved these changes

Assignees
No one assigned
Labels
component:integration-test component:marvin component:networking component:orchestration component:storage component:vpc Python Warning... Python code Ahead!
Projects
None yet
Milestone
4.20.0
Development

Successfully merging this pull request may close these issues.

Improve ConfigDrive to store network information
7 participants
@vishesh92 @blueorangutan @harikrishna-patnala @rohityadavcloud @DaanHoogland @weizhouapache @vladimirpetrov