0

I m trying to fragmentation an IPv4 packet using the below logic:

~after pkts ingress~

struct rte_port_ring_writer *p = port_out->h_port;

pool_direct = rte_mempool_lookup("MEMPOOL0");  
pool_indirect = rte_mempool_lookup("MEMPOOL1");

printf("before frag mempool size d %d in %d\n",rte_mempool_avail_count(pool_direct),rte_mempool_avail_count(pool_indirect));

struct rte_mbuf *frag_pkts[MAX_FRAG_SIZE];  
int out_pkts =    rte_ipv4_fragment_packet (m, frag_pkts, n_frags, ip_mtu,pool_direct, pool_indirect);

printf("after frag mempool size d %d in %d\n",rte_mempool_avail_count(pool_direct),rte_mempool_avail_count(pool_indirect));

if(out_pkts > 0)  
port_out->ops.f_tx_bulk(port_out->h_port,frag_pkts,RTE_LEN2MASK(out_pkts, uint64_t));  
else  
printf("frag failed\n");

rte_pktmbuf_free>(m);                       //free parent pkt

Now the problem here is the indirect mempool gets exhausted. As a result after few burst of packets the fragmentation fails due to -ENOMEM. I quite cannot understand why the PMD doesn't free and put back the mempool obj back to MEMPOOL1. Is it highly unlikely it is because of NIC ports being bounded to MEMPOOL0 and frag pkts from MEMPOOL1 being egressed.

Please find the log below for the above snippet which prints the available slots in direct (d) and indirect (in) mempools:

before frag mempool size d 2060457 in 2095988
after frag mempool size d 2060344 in 2095952
before frag mempool size d 2060361 in 2095945
after frag mempool size d 2060215 in 2095913
.
.
.
before frag mempool size d 2045013 in 0
after frag mempool size d 2045013 in 0
before frag mempool size d 2045013 in 0
after frag mempool size d 2045013 in 0
before frag mempool size d 2045013 in 0

I can see the direct mempool reduce and increase as packets ingress and drop/egress as expected. I can also confirm I receive the initial burst of fragmented packets equal to MEMPOOL1 size. Any inputs towards understanding the cause of the problem is much appreciated.

P.S: We had the same problem in dpdk17.11. We had to refractor the rte_ipv4_fragment_packet() to not use indirect chaining of frags instead just generate them.

Edit:
DPDK version - 20.11
Env - linux - centos 7
PMD - i40e - using bond in mode4
Pkt size - 128
MTU - 70

Mempools are created with rte_pktmbuf_pool_create(), thus with no SC/SP flag (defaults to MC/MP). Also always n_frags < MAX_FRAG_SIZE.

Thanks & Regards,
Vishal Mohan

12
  • please update the question with the following information 1) DPDK version 2) Linux or BSD or Windows 3) NIC PMD in use 4) Proc-info memopool result 5) please update the packet size of original packet m 5) is n_frags == MAX_FRAG_SIZE 6) ip_mtu value 7) what are parameter values used for creating direct and indirect mempool. @VishalMohan looking forward to the update Jul 13, 2021 at 13:49
  • Also, there is an example in DPDK ip_fragmentation which exercises the same API. Can you please confirm you are able to reproduce the same error. @VishalMohan looking forward to the update Jul 13, 2021 at 14:07
  • Hi @VipinVarghese, thank you for the response. I've updated the required information in the OP. Also will test the ip_frag example given by DPDK. Does using bond make any difference at all ? AFAIK bond pmd is only responsible for selecting a bond port and still the NIC PMD is responsible for egress and mbuf memory management. Please do correct me if this understanding is wrong Jul 14, 2021 at 8:16
  • there is gap in understanding, I believe you have my skype id please reach on skype for further discussion Jul 14, 2021 at 9:06
  • Hi @VishalMohan waiting for sync up from your end. Jul 14, 2021 at 15:08

1 Answer 1

1

DPDK API rte_ipv4_fragment_packet is used under both testpmd and DPDK example ip_fragementation. These are also included under the DPDK test suite which is run for each release too.

Based on the internal test and proper use of API for example Ip_fragementation the issue is not been able to reproduce. Hence the API leaking memory pool buffers are highly unlikely other than some special corner case (which is yet to be found).

Based on the code snippet analysis following could be the cause of mempool exhaust

  1. fail to free direct buffer after fragmentation
  2. fail to free one or more fragments from the indirect buffer when tx_burst fails.

[EDIT-1] based on the email update and comments, there is indeed no problem with DPDK API rte_ipv4_fragment_packet. The issue is from Application logic, with new behaviour as

  1. DPDK BOND PMD leads to mempool exhaustion with current code snippet
  2. DPDK BOND PMD has no issues with DPDK example with rte_ipv4_fragment_packet
  3. DPDK i40e PMD has an issue with the current code snippet
  4. DPDK i40e PMD has no issue with dpdk example rte_ipv4_fragment_packet

Hence the issue is with sample code snippet and usage and not DPDK API.

1
  • Thanks for the support Vipin. Yes everything is good with the examples. But not when used along with our app. There is no memleak for sure because post frag we tx the packet to bond port (Assuming pmd will free the mbuf). Will revise the app logic and update if any. Jul 20, 2021 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.