When VPP is killed (even with -9 option), operating system needs some time to clean resources, and the time it needs is not entirely deterministic. When CSIT starts another VPP instance in the next test (or executes bash commands to configure NICs using Linux drivers), the worst consequence is a PCI device still being bound to the old VPP instance.
This issue cannot be entirely prevented, as Linux does not give any guarantee on when PCI device is available again, but it can be mitigated by CSIT logic.
Such mitigation [0] has been merged during release testing, but some tests are still affected.
The best example is this [1], at least I believe the fact .1 device was not visible was due to the old VPP instance still being around.
I believe no current test is suffering from this anymore, but opening the ticket for rls2502 report purposes.
[0] 42022: feat(VPPUtil): Wait up to 10s to confirm VPP kill | https://gerrit.fd.io/r/c/csit/+/42022
[1] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2502-2n-grc/14/log.html.gz#s1-s1-s1-s1-s2-t7-k2-k4-k1-k1-k1-k1-k1
When VPP is killed (even with -9 option), operating system needs some time to clean resources, and the time it needs is not entirely deterministic. When CSIT starts another VPP instance in the next test (or executes bash commands to configure NICs using Linux drivers), the worst consequence is a PCI device still being bound to the old VPP instance.
This issue cannot be entirely prevented, as Linux does not give any guarantee on when PCI device is available again, but it can be mitigated by CSIT logic.
Such mitigation [0] has been merged during release testing, but some tests are still affected.
The best example is this [1], at least I believe the fact .1 device was not visible was due to the old VPP instance still being around.
I believe no current test is suffering from this anymore, but opening the ticket for rls2502 report purposes.
[0] 42022: feat(VPPUtil): Wait up to 10s to confirm VPP kill | https://gerrit.fd.io/r/c/csit/+/42022
[1] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2502-2n-grc/14/log.html.gz#s1-s1-s1-s1-s2-t7-k2-k4-k1-k1-k1-k1-k1