-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
None
This is affecting 3-node performance, usually causing MRR values to be significantly lower than PDR. Affects both 3n-icx and 3n-alt. Example trending with 2 workers [0], but happens also with only one worker [1]. Not seen yet on other NICs [2], so this issue is probably specific to mlx5_core driver used by dpdk plugin for cx6dx NIC.
Looking at show runtime [3], dpdk-input Vectors/Call stays below 256 (good MRR tests achieve nearly 512) while still producing (almost) two frames per call.
In MRR tests the queue from TG should always be full, so it is not clear what makes dpdk-input not read 256 from it.
[0] https://csit.fd.io/trending/#eNrdVMsKwjAQ_Jp6kYW-e_Jg7X9Imq420EdIorR-vWkpbAuKKCLUQ17MZiY7DNGmVXjUWO2cKHWS1PETUdjJCfZbu1ylhKABVhnwXPeMvvR4Fxcd1FUXgZAhxGEOHgc05XCyI2caoVZqIPEPA0lxMQtGQmTZE_JUh-qZQkYXrBhBBvVMZfkcqjopVqMWN6RS2wDh3LpBkMeX_KaXM3TqMcnGiq9ZJ4v_t27q8YV1gnc_Sd1jnVWn7m3rPkzdqqyj1EXZpmlVPf55fjJsWTVpRNkdGLFe9A
[1] https://csit.fd.io/trending/#eNrdUksOgyAUPI3dNC8R_LDqoq33aFBfqwkoAdpoT18wJk8X3XXVBb_MMMNMcH60eHOoTklxScQl4aJvw5Rk52NYXsZANoBUHliaPpAb1kxlO4FWUwG9yaHMa2ANoO_iKYxaOgRtbRTh1yjSPv1OkRDTzYR89SG-tCjpQjAjyKPbuOyfQ6y7lRpd_0aihgCEN6ENgliz1_ez2aBrRlEtjJ9VZ9r_r27NGKorqsMwWr38Pi7iVqrVo6g-gey10Q
[2] https://csit.fd.io/trending/#eNrlUssOgjAQ_Bq8mE0oD_HiQeQ_TGlXbRRY22rEr7cQkoWDN28e2qaZ2ZnOpM53Fo8Ob7soL6OijJLC6LBF6X4djicRpC0Y9QIRx2dMSOBWxOoOmvQVDGWwyWoQCtBfhltYtXQIjbWDSnIYVPTDLyQZoUvPyHcjHpAWJU8EN4Y8upnN8j3MOlnZoDNvZGpIwLgKfTAk1FLf9zRDp5BFNTJ-Vx7pPyhvChnKy6tV29lm_IF59QEyybD4
[3] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-mrr-daily-master-3n-icx/383/log.html.gz#s1-s1-s1-s2-s1-t1-k2-k9-k9-k10-k1-k1-k1-k12