We would need an improvement in the NAT plugin for the following STN setup:

Traffic:

Node 1 (192.168.16.2) -> Service (192.168.16.3:31093) translate to POD (10.1.2.138:31313)

Traffic flow:

Linux (192.168.16.2) -> VPP tap0 (VRF 0) -> VPP VRF 1 -> VXLAN (loop 0 – VRF 1)  – druhy node, ktory zda sa funguje spravne

Reply in the opposite direction.

The reply is not NATed correctly back to the service IP.

 

 

 

vpp# sh ver

vpp v18.07-rc0~486-g8008d7c built by root on 52d6286dd6a2 at Tue Jul 10 14:12:44 UTC 2018

 

pp# sh inter addr

GigabitEthernet0/8/0 (up):

  L3 192.168.16.2/24

local0 (dn):

loop0 (up):

  L2 bridge bd-id 1 idx 1 shg 1 bvi

  L3 192.168.30.1/24 ip4 table-id 1 fib-idx 1

tap0 (up):

  unnumbered, use GigabitEthernet0/8/0

  L3 192.168.16.2/24

tap1 (up):

  L3 10.2.1.2/32 ip4 table-id 1 fib-idx 1

tap2 (up):

  L3 10.2.1.41/32 ip4 table-id 1 fib-idx 1

tap3 (up):

  L3 10.2.1.42/32 ip4 table-id 1 fib-idx 1

vxlan_tunnel0 (up):

  L2 bridge bd-id 1 idx 1 shg 1 

 

vpp# sh nat44 static mappings

NAT44 static mappings:

tcp local 192.168.16.2:6443 external 10.96.0.1:443 vrf 0 self-twice-nat out2in-only

tcp local 192.168.16.2:12379 external 192.168.16.2:32379 vrf 0 self-twice-nat out2in-only

tcp local 192.168.16.2:12379 external 10.104.153.55:12379 vrf 0 self-twice-nat out2in-only

udp local 192.168.16.2:4789 external 192.168.16.2:4789 vrf 0 

 local 192.168.16.2 external 192.168.16.2 vrf 0 

 udp local 10.1.1.2:53 external 10.96.0.10:53 vrf 1 self-twice-nat out2in-only

tcp local 10.1.1.2:53 external 10.96.0.10:53 vrf 1 self-twice-nat out2in-only

tcp local 192.168.16.2:12379 external 192.168.16.3:32379 vrf 0 self-twice-nat out2in-only

tcp local 10.1.2.138:31313 external 10.111.143.231:31313 vrf 1 self-twice-nat out2in-only

tcp local 10.1.2.138:31313 external 192.168.16.3:31093 vrf 1 self-twice-nat out2in-only

tcp local 10.1.2.138:31313 external 192.168.16.2:31093 vrf 1 self-twice-nat out2in-only

tcp local 10.1.1.41:31313 external 10.97.139.73:31313 vrf 1 self-twice-nat out2in-only

tcp local 10.1.1.41:31313 external 192.168.16.3:30208 vrf 1 self-twice-nat out2in-only

tcp local 10.1.1.41:31313 external 192.168.16.2:30208 vrf 1 self-twice-nat out2in-only

vpp#

vpp#

 

 

Packet 10:

00:59:54:639506: virtio-input

  virtio: hw_if_index 2 next-index 4 vring 0 len 74

    hdr: flags 0x00 gso_type 0x00 hdr_len 0 gso_size 0 csum_start 0 csum_offset 0 num_buffers 1

00:59:54:639511: ethernet-input

  IP4: 00:00:00:00:00:02 -> 01:23:45:67:89:42

00:59:54:639514: ip4-input

  TCP: 192.168.16.2 -> 192.168.16.3

    tos 0x00, ttl 64, length 60, checksum 0x06f0

    fragment id 0x9276, flags DONT_FRAGMENT

  TCP: 32914 -> 31093

    seq. 0xf6c84863 ack 0x00000000

    flags 0x02 SYN, tcp header: 40 bytes

    window 29200, checksum 0x1a70

00:59:54:639518: nat44-ed-classify

  nat44-classify: next nat44-out2in

00:59:54:639520: nat44-ed-out2in

  NAT44_OUT2IN_FAST_PATH: sw_if_index 2, next index 1, session 12

00:59:54:639524: ip4-lookup

  fib 1 dpo-idx 6 flow hash: 0x00000000

  TCP: 192.168.16.2 -> 10.1.2.138

    tos 0x00, ttl 64, length 60, checksum 0xcb10

    fragment id 0x9276, flags DONT_FRAGMENT

  TCP: 32914 -> 31313

    seq. 0xf6c84863 ack 0x00000000

    flags 0x02 SYN, tcp header: 40 bytes

    window 29200, checksum 0xddb4

00:59:54:639526: ip4-rewrite

  tx_sw_if_index 3 dpo-idx 6 : ipv4 via 192.168.30.2 loop0: mtu:9000 1a2b3c4d5e021a2b3c4d5e010800 flow hash: 0x00000000

  00000000: 1a2b3c4d5e021a2b3c4d5e0108004500003c927640003f06cc10c0a810020a01

  00000020: 028a80927a51f6c8486300000000a0027210ddb40000020405b40402

00:59:54:639528: loop0-output

  loop0

  IP4: 1a:2b:3c:4d:5e:01 -> 1a:2b:3c:4d:5e:02

  TCP: 192.168.16.2 -> 10.1.2.138

    tos 0x00, ttl 63, length 60, checksum 0xcc10

    fragment id 0x9276, flags DONT_FRAGMENT

  TCP: 32914 -> 31313

    seq. 0xf6c84863 ack 0x00000000

    flags 0x02 SYN, tcp header: 40 bytes

    window 29200, checksum 0xddb4

00:59:54:639530: l2-input

  l2-input: sw_if_index 3 dst 1a:2b:3c:4d:5e:02 src 1a:2b:3c:4d:5e:01

00:59:54:639531: l2-fwd

  l2-fwd:   sw_if_index 3 dst 1a:2b:3c:4d:5e:02 src 1a:2b:3c:4d:5e:01 bd_index 1

00:59:54:639534: l2-output

  l2-output: sw_if_index 5 dst 1a:2b:3c:4d:5e:02 src 1a:2b:3c:4d:5e:01 data 08 00 45 00 00 3c 92 76 40 00 3f 06

00:59:54:639535: vxlan4-encap

  VXLAN encap to vxlan_tunnel0 vni 10

00:59:54:639537: ip4-rewrite

  tx_sw_if_index 1 dpo-idx 5 : ipv4 via 192.168.16.3 GigabitEthernet0/8/0: mtu:1500 0800271839ef0800279449d80800 flow hash: 0x00000002

  00000000: 0800271839ef0800279449d808004500006e00000000fd111c29c0a81002c0a8

  00000020: 10034b5e12b5005a00000800000000000a001a2b3c4d5e021a2b3c4d

00:59:54:639538: nat44-ed-in2out-output

  NAT44_IN2OUT_FAST_PATH: sw_if_index 3, next index 0, session 18

00:59:54:639541: GigabitEthernet0/8/0-output

  GigabitEthernet0/8/0

  IP4: 08:00:27:94:49:d8 -> 08:00:27:18:39:ef

  UDP: 192.168.16.2 -> 192.168.16.3

    tos 0x00, ttl 253, length 110, checksum 0x1c29

    fragment id 0x0000

  UDP: 14781 -> 4789

    length 90, checksum 0x0000

00:59:54:639542: GigabitEthernet0/8/0-tx

  GigabitEthernet0/8/0 tx queue 0

  buffer 0x10ca8: current data -50, length 124, free-list 0, clone-count 0, totlen-nifb 0, trace 0x6f

                  nated l2-hdr-offset 0 l3-hdr-offset 14

  PKT MBUF: port 65535, nb_segs 1, pkt_len 124

    buf_len 2176, data_len 124, ol_flags 0x0, data_off 78, phys_addr 0x10232a80

    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0

    rss 0x0 fdir.hi 0x0 fdir.lo 0x0

  IP4: 08:00:27:94:49:d8 -> 08:00:27:18:39:ef

  UDP: 192.168.16.2 -> 192.168.16.3

    tos 0x00, ttl 253, length 110, checksum 0x1c29

    fragment id 0x0000

  UDP: 14781 -> 4789

    length 90, checksum 0x0000

 

Packet 11:

00:59:54:640113: dpdk-input

  GigabitEthernet0/8/0 rx queue 0

  buffer 0x5e6d8: current data 14, length 110, free-list 0, clone-count 0, totlen-nifb 0, trace 0x70

                  ext-hdr-valid

                  l4-cksum-computed l4-cksum-correct l2-hdr-offset 0

  PKT MBUF: port 0, nb_segs 1, pkt_len 124

    buf_len 2176, data_len 124, ol_flags 0x0, data_off 128, phys_addr 0xd19b680

    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0

    rss 0x0 fdir.hi 0x0 fdir.lo 0x0

  IP4: 08:00:27:18:39:ef -> 08:00:27:94:49:d8

  UDP: 192.168.16.3 -> 192.168.16.2

    tos 0x00, ttl 253, length 110, checksum 0x1c29

    fragment id 0x0000

  UDP: 13453 -> 4789

    length 90, checksum 0x0000

00:59:54:640129: ip4-input

  UDP: 192.168.16.3 -> 192.168.16.2

    tos 0x00, ttl 253, length 110, checksum 0x1c29

    fragment id 0x0000

  UDP: 13453 -> 4789

    length 90, checksum 0x0000

00:59:54:640132: nat44-ed-out2in

  NAT44_OUT2IN_FAST_PATH: sw_if_index 1, next index 1, session 365

00:59:54:640133: ip4-lookup

  fib 0 dpo-idx 5 flow hash: 0x00000000

  UDP: 192.168.16.3 -> 192.168.16.2

    tos 0x00, ttl 253, length 110, checksum 0x1c29

    fragment id 0x0000

  UDP: 13453 -> 4789

    length 90, checksum 0x0000

00:59:54:640135: ip4-local

    UDP: 192.168.16.3 -> 192.168.16.2

      tos 0x00, ttl 253, length 110, checksum 0x1c29

      fragment id 0x0000

    UDP: 13453 -> 4789

      length 90, checksum 0x0000

00:59:54:640136: ip4-udp-lookup

  UDP: src-port 13453 dst-port 4789

00:59:54:640137: vxlan4-input

  VXLAN decap from vxlan_tunnel0 vni 10 next 1 error 0

00:59:54:640139: l2-input

  l2-input: sw_if_index 5 dst 1a:2b:3c:4d:5e:01 src 1a:2b:3c:4d:5e:02

00:59:54:640140: l2-fwd

  l2-fwd:   sw_if_index 5 dst 1a:2b:3c:4d:5e:01 src 1a:2b:3c:4d:5e:02 bd_index 1

00:59:54:640141: ip4-input

  TCP: 10.1.2.138 -> 192.168.16.2

    tos 0x00, ttl 63, length 60, checksum 0x5e87

    fragment id 0x0000, flags DONT_FRAGMENT

  TCP: 31313 -> 32914

    seq. 0x351e8e69 ack 0xf6c84864

    flags 0x12 SYN ACK, tcp header: 40 bytes

    window 28960, checksum 0x2274

00:59:54:640141: nat44-ed-classify

  nat44-classify: next nat44-out2in

00:59:54:640142: nat44-ed-out2in

  NAT44_OUT2IN_FAST_PATH: sw_if_index 3, next index 1, session 162

00:59:54:640143: ip4-lookup

  fib 0 dpo-idx 5 flow hash: 0x00000000

  TCP: 10.1.2.138 -> 192.168.16.2

    tos 0x00, ttl 63, length 60, checksum 0x5e87

    fragment id 0x0000, flags DONT_FRAGMENT

  TCP: 31313 -> 32914

    seq. 0x351e8e69 ack 0xf6c84864

    flags 0x12 SYN ACK, tcp header: 40 bytes

    window 28960, checksum 0x2274

00:59:54:640143: ip4-local

    TCP: 10.1.2.138 -> 192.168.16.2

      tos 0x00, ttl 63, length 60, checksum 0x5e87

      fragment id 0x0000, flags DONT_FRAGMENT

    TCP: 31313 -> 32914

      seq. 0x351e8e69 ack 0xf6c84864

      flags 0x12 SYN ACK, tcp header: 40 bytes

      window 28960, checksum 0x2274

00:59:54:640143: ip4-punt

    TCP: 10.1.2.138 -> 192.168.16.2

      tos 0x00, ttl 63, length 60, checksum 0x5e87

      fragment id 0x0000, flags DONT_FRAGMENT

    TCP: 31313 -> 32914

      seq. 0x351e8e69 ack 0xf6c84864

      flags 0x12 SYN ACK, tcp header: 40 bytes

      window 28960, checksum 0x2274

00:59:54:640144: stn-ip4-punt

  dst_address: 192.168.16.2

  rule:

    rule_index: 0

    address: 192.168.16.2

    iface: tap0 (2)

    next_node: tap0-output (518)

00:59:54:640146: tap0-output

  tap0

  IP4: 00:00:00:00:00:01 -> 00:00:00:00:00:02

  TCP: 10.1.2.138 -> 192.168.16.2

    tos 0x00, ttl 63, length 60, checksum 0x5e87

    fragment id 0x0000, flags DONT_FRAGMENT

  TCP: 31313 -> 32914

    seq. 0x351e8e69 ack 0xf6c84864

    flags 0x12 SYN ACK, tcp header: 40 bytes

    window 28960, checksum 0x2274