Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[VPP-2125] Vpp Ikev2 initiator crashed after "vppctl ikev2 initiate sa-init pr1" when enable multi threads #3586

Open
vvalderrv opened this issue Feb 2, 2025 · 3 comments
Open

[VPP-2125] Vpp Ikev2 initiator crashed after "vppctl ikev2 initiate sa-init pr1" when enable multi threads #3586

vvalderrv opened this issue Feb 2, 2025 · 3 comments

Comments

@vvalderrv
Copy link
Contributor

Description

Vpp Ikev2 initiator crashed after  exec"vppctl ikev2 initiate sa-init pr1" to establish ipsec tunnel,when enable multi-threads while Single multi-threads is works.

PLease see the attachment core dump file.

 

 Ikev2 initiator /var/log/syslog output:

Oct  9 14:23:17 j-lb14 systemd[1]: Starting vector packet processing engine...

Oct  9 14:23:17 j-lb14 systemd[1]: Started vector packet processing engine.

Oct  9 14:23:18 j-lb14 vpp[9433]: vpp[9433]: dpdk_config:1363: socket-mem argument is deprecated

Oct  9 14:23:18 j-lb14 vpp[9433]: dpdk_config:1363: socket-mem argument is deprecated

Oct  9 14:23:18 j-lb14 kernel: [158210.472412] vfio-pci 0000:41:00.0: vfio_ecap_init: hiding ecap 0x19@0x1d0

Oct  9 14:23:18 j-lb14 kernel: [158210.472419] vfio-pci 0000:41:00.0: vfio_ecap_init: hiding ecap 0x25@0x200

Oct  9 14:23:18 j-lb14 kernel: [158210.472422] vfio-pci 0000:41:00.0: vfio_ecap_init: hiding ecap 0x26@0x210

Oct  9 14:23:18 j-lb14 kernel: [158210.472426] vfio-pci 0000:41:00.0: vfio_ecap_init: hiding ecap 0x27@0x250

Oct  9 14:23:19 j-lb14 kernel: [158210.880789] vfio-pci 0000:c4:00.0: vfio_ecap_init: hiding ecap 0x19@0x1d0

Oct  9 14:23:19 j-lb14 kernel: [158210.880794] vfio-pci 0000:c4:00.0: vfio_ecap_init: hiding ecap 0x25@0x200

Oct  9 14:23:19 j-lb14 kernel: [158210.880797] vfio-pci 0000:c4:00.0: vfio_ecap_init: hiding ecap 0x26@0x210

Oct  9 14:23:19 j-lb14 kernel: [158210.880800] vfio-pci 0000:c4:00.0: vfio_ecap_init: hiding ecap 0x27@0x250

Oct  9 14:23:19 j-lb14 vpp[9433]: vat-plug/load: vat_plugin_register: idpf plugin not loaded...

Oct  9 14:28:49 j-lb14 vpp[9433]: received signal SIGSEGV, PC 0x7f68babf0e63, faulting address 0x1ea

Oct  9 14:28:49 j-lb14 vpp[9433]: #0  0x00007f68bad0bc72 0x7f68bad0bc72

Oct  9 14:28:49 j-lb14 vpp[9433]: #1  0x00007f68ba8ed520 0x7f68ba8ed520

Oct  9 14:28:49 j-lb14 vpp[9433]: #2  0x00007f68babf0e63 clib_mem_heap_free + 0x73

Oct  9 14:28:49 j-lb14 vpp[9433]: #3  0x00007f68b1fac110 0x7f68b1fac110

Oct  9 14:28:49 j-lb14 vpp[9433]: #4  0x00007f68b1fc5fff 0x7f68b1fc5fff

Oct  9 14:28:49 j-lb14 vpp[9433]: #5  0x00007f68baca1807 0x7f68baca1807

Oct  9 14:28:49 j-lb14 vpp[9433]: #6  0x00007f68bac2ffd8 0x7f68bac2ffd8

Oct  9 14:28:49 j-lb14 systemd[1]: vpp.service: Main process exited, code=dumped, status=6/ABRT

Oct  9 14:28:49 j-lb14 systemd[1]: vpp.service: Failed with result 'core-dump'.

Oct  9 14:28:49 j-lb14 systemd[1]: vpp.service: Consumed 5min 42.391s CPU time.

Oct  9 14:28:50 j-lb14 systemd[1]: vpp.service: Scheduled restart job, restart counter is at 2.

Oct  9 14:28:50 j-lb14 systemd[1]: Stopped vector packet processing engine.

Oct  9 14:28:50 j-lb14 systemd[1]: vpp.service: Consumed 5min 42.391s CPU time.

Assignee

Neale Ranns

Reporter

John

Comments

  • JIRAUSER16685 (Thu, 14 Nov 2024 11:59:18 +0000): I am getting this crash randomly in single thread as well. Please help in providing workaround
  • JIRAUSER16685 (Thu, 14 Nov 2024 11:57:32 +0000):

    vpp is crashing randomly with below bt on ikev2 sa init at initiator side:

Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x00007ffff7ef6e63 in clib_mem_heap_free () from /lib/x86_64-linux-gnu/libvppinfra.so.24.06

(gdb)

(gdb)

(gdb)

(gdb)

(gdb)

(gdb) bt

#0  0x00007ffff7ef6e63 in clib_mem_heap_free () from /lib/x86_64-linux-gnu/libvppinfra.so.24.06

#1  0x00007ffff5f9ad9b in ?? () from /usr/lib/x86_64-linux-gnu/vpp_plugins/ikev2_plugin.so

#2  0x00007ffff5fb4fff in ?? () from /usr/lib/x86_64-linux-gnu/vpp_plugins/ikev2_plugin.so

#3  0x00007ffff603b807 in ?? () from /lib/x86_64-linux-gnu/libvlib.so.24.06

#4  0x00007ffff7f35fd8 in clib_calljmp () from /lib/x86_64-linux-gnu/libvppinfra.so.24.06

#5  0x00007fffeea00d50 in ?? ()

#6  0x00007ffff60323da in ?? () from /lib/x86_64-linux-gnu/libvlib.so.24.06

#7  0x0000000100000001 in ?? ()

#8  0x0000000200000001 in ?? ()

#9  0x0000000000000000 in ?? ()

 

Tunnel Configuration:

Initiator:

VPP Version: 24.06

Interface:

vpp# show interface

              Name               Idx    State  MTU (L3/IP4/IP6/MPLS)     Counter          Count

SECGW-ACCESS_RAN                  1      up          9000/0/0/0

SECGW-CORE_RAN                    2      up          9000/0/0/0

ipip0                             3     down         9000/0/0/0

local0                            0     down          0/0/0/0

vpp# show interface addr

SECGW-ACCESS_RAN (up):

  L3 7.7.7.10/24

SECGW-CORE_RAN (up):

  L3 192.168.10.1/24

ipip0 (dn):

  unnumbered, use SECGW-CORE_RAN

  L3 192.168.10.1/24

local0 (dn):

profile pr1

  auth-method shared-key-mic auth data Vpp123

  local id-type ip4-addr data 192.168.10.1

  remote id-type ip4-addr data 192.168.10.2

  local traffic-selector addr 7.7.7.1 - 7.7.7.255 port 0 - 65535 protocol 0

  remote traffic-selector addr 5.5.5.1 - 5.5.5.255 port 0 - 65535 protocol 0

  protected tunnel ipip0

  responder SECGW-CORE_RAN 192.168.10.2

  ike-crypto-alg aes-gcm-16 256 ike-integ-alg none ike-dh modp-2048

  esp-crypto-alg aes-gcm-16 256 esp-integ-alg none

  lifetime 0 jitter 0 handover 0 maxdata 0

Responder:

VPP Version: 22.06

             Name               Idx    State  MTU (L3/IP4/IP6/MPLS)    

SECGW-ACCESS                      2      up          9000/0/0/0     

                                                                    

                                                                    

                                                                    

                                                                    

                                                                    

SECGW-CORE                        1      up          9000/0/0/0     

                                                                    

                                                                    

                                                                    

                                                                    

                                                                    

                                                                    

ipip1                             3      up          9000/0/0/0     

                                                                    

                                                                    

                                                                    

local0                            0     down          0/0/0/0

vpp# show interface addr

SECGW-ACCESS (up):

  L3 5.5.5.20/24

SECGW-CORE (up):

  L3 192.168.10.2/24

ipip1 (up):

  unnumbered, use SECGW-CORE

  L3 192.168.10.2/24

local0 (dn):

vpp# show ikev2 profile

profile pr1

  auth-method shared-key-mic auth data Vpp123

  local id-type ip4-addr data 192.168.10.2

  remote id-type ip4-addr data 192.168.10.1

  local traffic-selector addr 5.5.5.0 - 5.5.5.255 port 0 - 65535 protocol 0

  remote traffic-selector addr 7.7.7.0 - 7.7.7.255 port 0 - 65535 protocol 0

  protected tunnel ipip1

  lifetime 0 jitter 0 handover 0 maxdata 0

Please find attach PCAP at Responder side and vpp/startup.conf

 

I am getting same crash with single core also. Can you please share working config if available? 

 

Thanks,

Chirag Jain

Original issue: https://jira.fd.io/browse/VPP-2125
@vvalderrv
Copy link
Contributor Author

I am getting this crash randomly in single thread as well. Please help in providing workaround

@vvalderrv
Copy link
Contributor Author

vpp is crashing randomly with below bt on ikev2 sa init at initiator side:
Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ef6e63 in clib_mem_heap_free () from /lib/x86_64-linux-gnu/libvppinfra.so.24.06
(gdb)
(gdb)
(gdb)
(gdb)
(gdb)
(gdb) bt
#0  0x00007ffff7ef6e63 in clib_mem_heap_free () from /lib/x86_64-linux-gnu/libvppinfra.so.24.06
#1  0x00007ffff5f9ad9b in ?? () from /usr/lib/x86_64-linux-gnu/vpp_plugins/ikev2_plugin.so
#2  0x00007ffff5fb4fff in ?? () from /usr/lib/x86_64-linux-gnu/vpp_plugins/ikev2_plugin.so
#3  0x00007ffff603b807 in ?? () from /lib/x86_64-linux-gnu/libvlib.so.24.06
#4  0x00007ffff7f35fd8 in clib_calljmp () from /lib/x86_64-linux-gnu/libvppinfra.so.24.06
#5  0x00007fffeea00d50 in ?? ()
#6  0x00007ffff60323da in ?? () from /lib/x86_64-linux-gnu/libvlib.so.24.06
#7  0x0000000100000001 in ?? ()
#8  0x0000000200000001 in ?? ()
#9  0x0000000000000000 in ?? ()

 

Tunnel Configuration:

Initiator:
VPP Version: 24.06
Interface:
vpp# show interface
              Name               Idx    State  MTU (L3/IP4/IP6/MPLS)     Counter          Count
SECGW-ACCESS_RAN                  1      up          9000/0/0/0
SECGW-CORE_RAN                    2      up          9000/0/0/0
ipip0                             3     down         9000/0/0/0
local0                            0     down          0/0/0/0
vpp# show interface addr
SECGW-ACCESS_RAN (up):
  L3 7.7.7.10/24
SECGW-CORE_RAN (up):
  L3 192.168.10.1/24
ipip0 (dn):
  unnumbered, use SECGW-CORE_RAN
  L3 192.168.10.1/24
local0 (dn):

profile pr1
  auth-method shared-key-mic auth data Vpp123
  local id-type ip4-addr data 192.168.10.1
  remote id-type ip4-addr data 192.168.10.2
  local traffic-selector addr 7.7.7.1 - 7.7.7.255 port 0 - 65535 protocol 0
  remote traffic-selector addr 5.5.5.1 - 5.5.5.255 port 0 - 65535 protocol 0
  protected tunnel ipip0
  responder SECGW-CORE_RAN 192.168.10.2
  ike-crypto-alg aes-gcm-16 256 ike-integ-alg none ike-dh modp-2048
  esp-crypto-alg aes-gcm-16 256 esp-integ-alg none
  lifetime 0 jitter 0 handover 0 maxdata 0

Responder:
VPP Version: 22.06

             Name               Idx    State  MTU (L3/IP4/IP6/MPLS)    
SECGW-ACCESS                      2      up          9000/0/0/0     
                                                                    
                                                                    
                                                                    
                                                                    
                                                                    
SECGW-CORE                        1      up          9000/0/0/0     
                                                                    
                                                                    
                                                                    
                                                                    
                                                                    
                                                                    
ipip1                             3      up          9000/0/0/0     
                                                                    
                                                                    
                                                                    
local0                            0     down          0/0/0/0

vpp# show interface addr
SECGW-ACCESS (up):
  L3 5.5.5.20/24
SECGW-CORE (up):
  L3 192.168.10.2/24
ipip1 (up):
  unnumbered, use SECGW-CORE
  L3 192.168.10.2/24
local0 (dn):
vpp# show ikev2 profile
profile pr1
  auth-method shared-key-mic auth data Vpp123
  local id-type ip4-addr data 192.168.10.2
  remote id-type ip4-addr data 192.168.10.1
  local traffic-selector addr 5.5.5.0 - 5.5.5.255 port 0 - 65535 protocol 0
  remote traffic-selector addr 7.7.7.0 - 7.7.7.255 port 0 - 65535 protocol 0
  protected tunnel ipip1
  lifetime 0 jitter 0 handover 0 maxdata 0

Please find attach PCAP at Responder side and vpp/startup.conf

@vvalderrv
Copy link
Contributor Author

Hi Neale Ranns ,John 

 

I am getting same crash with single core also. Can you please share working config if available? 

 

Thanks,

Chirag Jain

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vvalderrv