There are multiple different failure messages all hinting at the same root cause:

2021/06/18 14:01:29:125 notice     dpdk           EAL: Using IOMMU type 1 (Type 1)

2021/06/18 14:01:29:125 notice     dpdk           EAL: Probe PCI driver: net_i40e_vf (8086:154c) device: 0000:91:02.0 (socket 1)

2021/06/18 14:01:29:125 notice     dpdk           EAL: Releasing PCI mapped resource for 0000:91:02.0

2021/06/18 14:01:29:125 notice     dpdk           EAL: Calling pci_unmap_resource for 0000:91:02.0 at 0x6309380000

2021/06/18 14:01:29:125 notice     dpdk           EAL: Calling pci_unmap_resource for 0000:91:02.0 at 0x6309390000

2021/06/18 14:01:29:125 notice     dpdk           EAL: Using IOMMU type 1 (Type 1)

2021/06/18 14:01:29:125 notice     dpdk           EAL: Probe PCI driver: net_iavf (8086:154c) device: 0000:91:02.0 (socket 1)

2021/06/18 14:01:29:125 notice     dpdk           EAL: Cannot open /dev/vfio/151: Device or resource busy

2021/06/18 14:01:29:125 notice     dpdk           EAL: Failed to open VFIO group 151

2021/06/18 14:01:29:125 notice     dpdk           EAL: Requested device 0000:91:03.2 cannot be used

2021/06/18 14:01:29:125 notice     dpdk           iavf_dev_init_vlan(): Failed to update vlan offload

2021/06/18 14:01:29:125 notice     dpdk           iavf_dev_configure(): configure VLAN failed: -95

Here we can see that the initialization of 0000:91:02.0 was successful using the iavf PMD (judging from the fact that net_iavf was called), but 0000:91:03.2 couldn't be used because of "Cannot open /dev/vfio/151: Device or resource busy". For reference:

localadmin@s56-t14-sut1:~$ ll /sys/bus/pci/devices/0000\:91\:03.2/iommu_group

lrwxrwxrwx 1 root root 0 Jun 21 07:37 /sys/bus/pci/devices/0000:91:03.2/iommu_group -> ../../../../kernel/iommu_groups/151/

In another run, we had these failures:

2021/06/16 22:03:50:782 notice     dpdk           EAL: VFIO support initialized

2021/06/16 22:03:50:782 notice     dpdk           EAL: Using IOMMU type 1 (Type 1)

2021/06/16 22:03:50:782 notice     dpdk           EAL: Getting a vfio_dev_fd for 0000:91:03.3 failed     # a different failure

2021/06/16 22:03:50:782 notice     dpdk           EAL: Requested device 0000:91:03.3 cannot be used

2021/06/16 22:03:50:782 notice     dpdk           EAL: Using IOMMU type 1 (Type 1)

2021/06/16 22:03:50:782 notice     dpdk           EAL: Could not install memory event callback for VFIO

2021/06/16 22:03:50:782 notice     dpdk           EAL: Requested device 0000:91:03.4 cannot be used      # yet another failure

2021/06/16 22:03:50:782 notice     dpdk           EAL: Bus (pci) probe failed.