Medium When capturing VPP packets on the output queues like VirtualEthernet0/0/3-output and BondEthernet0-output, VPP does not capture all the packets and stops capturing packets after 50 captured packets unexpectedly.
I enabled packet tracing for 1000 packets, but only 50 packets were captured.
$ vppctl clear counters $ vppctl clear trace $ vppctl trace add dpdk-input 1000 $ vppctl trace add VirtualEthernet0/0/3-output 1000 $ vppctl trace add vhost-user-input 1000 $ vppctl trace add BondEthernet0-output 1000
Also, ICMP packets were not captured at all when VPP forwards ICMP packets.
$ vppctl show trace | grep -i icmp $
Here is more info:
$ vppctl show interface Name Idx State Counter Count BondEthernet0 1 up rx packets 1763 rx bytes 295391 tx packets 894 tx bytes 672159 drops 863 BondEthernet0.2809 4 up rx packets 896 rx bytes 226695 tx packets 894 tx bytes 672159 BondEthernet0.2808 6 up rx packets 2 rx bytes 244 BondEthernet0.2811 8 up rx packets 2 rx bytes 244 TenGigabitEtherneta/0/0 2 bond-slave TenGigabitEtherneta/0/1 3 bond-slave VirtualEthernet0/0/0 5 up rx packets 871 rx bytes 665973 tx packets 873 tx bytes 220493 VirtualEthernet0/0/1 7 up tx packets 2 tx bytes 236 VirtualEthernet0/0/2 9 up tx packets 2 tx bytes 236 VirtualEthernet0/0/3 10 up rx packets 23 rx bytes 2610 tx packets 30 tx bytes 3834 local0 0 down $ vppctl show vhost Virtio vhost-user interfaces Global: coalesce frames 32 time 1e-3 Interface: VirtualEthernet0/0/0 (ifindex 4) virtio_net_hdr_sz 12 features mask (0xffffffffffffffff): features (0x140008000): VIRTIO_NET_F_MRG_RXBUF (15) VHOST_USER_F_PROTOCOL_FEATURES (30) VIRTIO_F_VERSION_1 (32) protocol features (0x3) VHOST_USER_PROTOCOL_F_MQ (0) VHOST_USER_PROTOCOL_F_LOG_SHMFD (1) socket filename /tmp/39c84881-1d98-473e-a6df-b655cafa6fba type client errno "Success" rx placement: thread 0 on vring 1 tx placement: lock-free thread 0 on vring 0 Memory regions (total 3) region fd guest_phys_addr memory_size userspace_addr mmap_offset mmap_addr ====== ===== ================== ================== ================== ================== ================== 0 25 0x0000000100000000 0x0000000040000000 0x00007f7f88200000 0x00000000c0000000 0x00002aab6ac00000 1 30 0x0000000000000000 0x00000000000a0000 0x00007f7ec8200000 0x0000000000000000 0x00002aabaae00000 2 34 0x00000000000c0000 0x00000000bff40000 0x00007f7ec82c0000 0x00000000000c0000 0x00002aafebec0000 Virtqueue 0 (TX) qsz 256 last_avail_idx 59703 last_used_idx 59703 avail.flags 1 avail.idx 59959 used.flags 1 used.idx 59703 kickfd 31 callfd 32 errfd -1 Virtqueue 1 (RX) qsz 256 last_avail_idx 10855 last_used_idx 10855 avail.flags 1 avail.idx 10855 used.flags 1 used.idx 10855 kickfd 33 callfd 35 errfd -1 Interface: VirtualEthernet0/0/1 (ifindex 5) virtio_net_hdr_sz 12 features mask (0xffffffffffffffff): features (0x140008000): VIRTIO_NET_F_MRG_RXBUF (15) VHOST_USER_F_PROTOCOL_FEATURES (30) VIRTIO_F_VERSION_1 (32) protocol features (0x3) VHOST_USER_PROTOCOL_F_MQ (0) VHOST_USER_PROTOCOL_F_LOG_SHMFD (1) socket filename /tmp/d0a5f19e-76e9-475a-a379-23b47c10e522 type client errno "Success" rx placement: thread 0 on vring 1 tx placement: lock-free thread 0 on vring 0 Memory regions (total 3) region fd guest_phys_addr memory_size userspace_addr mmap_offset mmap_addr ====== ===== ================== ================== ================== ================== ================== 0 29 0x0000000100000000 0x0000000040000000 0x00007f7f88200000 0x00000000c0000000 0x00002aae2b600000 1 41 0x0000000000000000 0x00000000000a0000 0x00007f7ec8200000 0x0000000000000000 0x00002aae6b800000 2 44 0x00000000000c0000 0x00000000bff40000 0x00007f7ec82c0000 0x00000000000c0000 0x00002aae6bac0000 Virtqueue 0 (TX) qsz 256 last_avail_idx 50362 last_used_idx 50362 avail.flags 1 avail.idx 50618 used.flags 1 used.idx 50362 kickfd 28 callfd 42 errfd -1 Virtqueue 1 (RX) qsz 256 last_avail_idx 25130 last_used_idx 25130 avail.flags 1 avail.idx 25130 used.flags 1 used.idx 25130 kickfd 43 callfd 45 errfd -1 Interface: VirtualEthernet0/0/2 (ifindex 6) virtio_net_hdr_sz 12 features mask (0xffffffffffffffff): features (0x140008000): VIRTIO_NET_F_MRG_RXBUF (15) VHOST_USER_F_PROTOCOL_FEATURES (30) VIRTIO_F_VERSION_1 (32) protocol features (0x3) VHOST_USER_PROTOCOL_F_MQ (0) VHOST_USER_PROTOCOL_F_LOG_SHMFD (1) socket filename /tmp/e0fb3cc8-d9e9-4051-93b4-c26a70034137 type client errno "Success" rx placement: thread 0 on vring 1 tx placement: lock-free thread 0 on vring 0 Memory regions (total 3) region fd guest_phys_addr memory_size userspace_addr mmap_offset mmap_addr ====== ===== ================== ================== ================== ================== ================== 0 27 0x0000000100000000 0x0000000040000000 0x00007f7f88200000 0x00000000c0000000 0x00002aac6b000000 1 36 0x0000000000000000 0x00000000000a0000 0x00007f7ec8200000 0x0000000000000000 0x00002aacab200000 2 39 0x00000000000c0000 0x00000000bff40000 0x00007f7ec82c0000 0x00000000000c0000 0x00002aacab4c0000 Virtqueue 0 (TX) qsz 256 last_avail_idx 58902 last_used_idx 58902 avail.flags 1 avail.idx 59158 used.flags 1 used.idx 58902 kickfd 26 callfd 37 errfd -1 Virtqueue 1 (RX) qsz 256 last_avail_idx 22436 last_used_idx 22436 avail.flags 1 avail.idx 22436 used.flags 1 used.idx 22436 kickfd 38 callfd 40 errfd -1 Interface: VirtualEthernet0/0/3 (ifindex 7) virtio_net_hdr_sz 12 features mask (0xffffffffffffffff): features (0x50008000): VIRTIO_NET_F_MRG_RXBUF (15) VIRTIO_F_INDIRECT_DESC (28) VHOST_USER_F_PROTOCOL_FEATURES (30) protocol features (0x3) VHOST_USER_PROTOCOL_F_MQ (0) VHOST_USER_PROTOCOL_F_LOG_SHMFD (1) socket filename /tmp/94c686a5-01b7-4050-a5f5-eeed0071a27f type client errno "Success" rx placement: thread 0 on vring 1 tx placement: lock-free thread 0 on vring 0 Memory regions (total 2) region fd guest_phys_addr memory_size userspace_addr mmap_offset mmap_addr ====== ===== ================== ================== ================== ================== ================== 0 49 0x0000000000000000 0x00000000000a0000 0x00007fde05c00000 0x0000000000000000 0x00002aaf2bc00000 1 50 0x00000000000c0000 0x000000001ff40000 0x00007fde05cc0000 0x00000000000c0000 0x00002aaf2bec0000 Virtqueue 0 (TX) qsz 256 last_avail_idx 28222 last_used_idx 28222 avail.flags 0 avail.idx 28381 used.flags 1 used.idx 28222 kickfd 51 callfd 52 errfd -1 Virtqueue 1 (RX) qsz 256 last_avail_idx 24807 last_used_idx 24807 avail.flags 1 avail.idx 24807 used.flags 1 used.idx 24807 kickfd 47 callfd 53 errfd -1
Only 50 packets were captured:
$ vppctl show trace | grep ^Packet Packet 1 Packet 2 Packet 3 Packet 4 Packet 5 Packet 6 Packet 7 Packet 8 Packet 9 Packet 10 Packet 11 Packet 12 Packet 13 Packet 14 Packet 15 Packet 16 Packet 17 Packet 18 Packet 19 Packet 20 Packet 21 Packet 22 Packet 23 Packet 24 Packet 25 Packet 26 Packet 27 Packet 28 Packet 29 Packet 30 Packet 31 Packet 32 Packet 33 Packet 34 Packet 35 Packet 36 Packet 37 Packet 38 Packet 39 Packet 40 Packet 41 Packet 42 Packet 43 Packet 44 Packet 45 Packet 46 Packet 47 Packet 48 Packet 49 Packet 50 $ vppctl show trace | grep ^Packet | wc -l 50
VPP version info:
$ vppctl show version verbose
Version: v17.01.4-release
Compiled by: root
Compile host: 9df7287b0025
Compile date: Wed May 17 11:15:15 UTC 2017
Compile location: /home/jenkins/workspace/vpp-merge-1701-centos7
Compiler: GCC 4.8.5 20150623 (Red Hat 4.8.5-11)
Current PID: 1400486
DPDK Version: DPDK 16.11.0
DPDK EAL init args: -c 1 -n 4 --vdev eth_bond0,mode=2,slave=0000:0a:00.0,slave=0000:0a:00.1,xmit_policy=l34 --huge-dir /run/vpp/hugepages --file-prefix vpp -w 0000:0a:00.0 -w 0000:0a:00.1 --master-lcore 0 --socket-mem 1024,0