FD.io CSIT_VIRL phased migration proposal.
------------------------------------------
Goal: migrate away from CSIT_VIRL test platform.
Approach: Move needed CSIT_VIRL tests to VPP_make_test framework and new VPP_Device functional tests run on physical servers. Existing VIRL test cases have been evaluated against VPP make_test tests and gaps prioritized. See spreadsheet [1] for reference.
Phasing (draft proposal):
- Remove from CSIT_VIRL remaining duplicate tests. - Tests identified as fully covered in existing VPP_make_test. - Before removing, check that all those tests are run per VPP patchset.
- Develop P0 MUST_HAVE tests in VPP_make_test. - Code should be pushed into vpp master branch into /test directory. - Check with VPP_make_test maintainer (i.e. Klement) if separate sub-directory should be created. - Once merged these tests MUST be executed per VPP patch.
- Remove from VIRL duplicate P0 tests.
- Develop P1 SHOULD_HAVE tests in VPP_make_test. - Once merged these tests SHOULD be executed per VPP patch. - Group tests into sets and execute in parallel to reduce job execution time.
- Remove from VIRL duplicate P1 tests. - Consider removing VIRL tests from per VPP patch tests.
- Develop P2 NICE TO_HAVE tests in VPP_make_test. - Once merged these tests COULD be executed per VPP patch.
- Remove from VIRL duplicate P2 tests. - Consider removing VIRL tests from per VPP patch tests.
- Move VIRL vhost tests to VPP_Device functional physical tests. - See proposed VPP_Device test design [2]. - Remove VIRL tests from per VPP patch tests. - Repurpose VIRL hosts to parallelise VPP_make_test. References: [1] vpp-csit-verify-virl-test-gaps, https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit?usp=sharing [2] https://lists.fd.io/g/csit-dev/message/2771