Uploaded image for project: 'vpp'
  1. vpp
  2. VPP-2092

vpp memory statistics collection starts to lag with huge number of allocations

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: High High
    • None
    • 23.06
    • VPPInfra
    • None

      copy from https://lists.fd.io/g/vpp-dev/topic/103039486

      internal_mallinfo in dlmalloc.c has complexity of O(memory chunks count), because it calculates stats by doing linked list iteration over all allocated objects. And in our case (vpp-upf plugin) we have millions of allocated objects.
       
      In our load test mspace_mallinfo hangs upg-vpp every 10 seconds for ~0.18 seconds
      with 4.5 millions of allocated objects/ This makes is so vpp stopped for 1.8% of the time and results in >1% packetloss even when vpp is not fully loaded.
      With workaround (removed stats collection) packetloss is 0.

      Screenshot of hotspot performance tool with blue parts being mspace_mallinfo

      Most probably this issue affects more deployments, but not as noticeable.

            dbarach Dave Barach
            mogaika Vladimir Zhigulin
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: