VPP 19.08.1 seems to die intermittently with the following in the logs:

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server [-] [Errno -1] vac_read failed.  It is likely that VPP died.: VppTransportShmemIOError: [Errno -1] vac_read failed.  It is likely that VPP died.

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server Traceback (most recent call last):

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server   File "/usr/local/lib/python2.7/dist-packages/networking_vpp/agent/vpp.py", line 383, in call_vpp

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server     t = func_call(*args, **kwargs)

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server   File "/usr/lib/python2.7/dist-packages/vpp_papi/vpp_papi.py", line 90, in _call_

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server     return self._func(**kwargs)

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server   File "/usr/lib/python2.7/dist-packages/vpp_papi/vpp_papi.py", line 422, in f

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server     return self._call_vpp(i, msg, multipart, **kwargs)

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server   File "/usr/lib/python2.7/dist-packages/vpp_papi/vpp_papi.py", line 648, in _call_vpp

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server     msg = self.transport.read()

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server   File "/usr/lib/python2.7/dist-packages/vpp_papi/vpp_transport_shmem.py", line 127, in read

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server     raise VppTransportShmemIOError(rv, strerror)

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server VppTransportShmemIOError: [Errno -1] vac_read failed.  It is likely that VPP died.

2019-10-18 14:57:09.000 1164 ERROR networking_vpp.agent.server

2019-10-18 14:57:09.002 1164 DEBUG vpp_papi.vpp_papi.VPPApiClient [-] Cleaning up VPP on exit vpp_atexit /usr/lib/python2.7/dist-packages/vpp_papi/vpp_papi.py:67

2019-10-18 14:57:10.006 1164 WARNING vpp_papi.vpp_papi.VPPApiClient [-] VPP API client:: vl_client_disconnect:300: peer unresponsive, give up

It was first reported by SURFnet who are using openstack+VPP+networking-vpp to run their NFVs in their datacenter. I asked them to upgrade to the latest VPP and networking-vpp. Well, they have hit the issue again with VPP/networking-vpp 19.08.1. In fact, I too have seen it a few times during my tests with VPP 19.08.1. It just seems to happen intermittently and I think there’s a memory leak somewhere of some sort.