TypeError: 'NoneType' object is not subscriptable

[ralonsoh]

This error is randomly happening in the Neutron functional CI (e.g.: [1]). The problem is that I'm not sure exactly where in the code is this problem because the pyroute2 calls are done via a root privileged daemon that only returns the error, not the trace. I've tried to reproduce this issue locally not calling pryoute2 library inside a privsep context (that calls the method remotely via this daemon) but I was not able.

This issue started happening on Feb 21 [2], so I've tried using different patches since this day. I've identified the culprit: 264c80c. When using this patch, the issue starts appearing in our CI.

[1]https://8011428c57c656673612-eb8cb59c110a9d33c33aa832e5c84dda.ssl.cf5.rackcdn.com/942859/5/check/neutron-functional-with-pyroute2-master-5/c8cbaf5/testr_results.html
[2]https://bugs.launchpad.net/neutron/+bug/2100261

[svinota]

Thanks for the debug info, it would be hard to find otherwise. Investigating

[svinota]

Looks like the error comes either from https://opendev.org/openstack/oslo.privsep/src/commit/959f01e9996873ffb863285d968faba68709d1ac/oslo_privsep/daemon.py#L480 , or from https://opendev.org/openstack/oslo.privsep/src/commit/959f01e9996873ffb863285d968faba68709d1ac/oslo_privsep/daemon.py#L508

Either way e.args is None, and it means that it is not a timeout, where result[2] would be an empty string (which is not much more correct, but wouldn't fail) (see https://opendev.org/openstack/oslo.privsep/src/commit/959f01e9996873ffb863285d968faba68709d1ac/oslo_privsep/comm.py#L126)

Ok, I'll try to see what can come from pyroute2:s .close() with .args == None.

[svinota]

@ralonsoh , can we run the CI against a custom pyroute2 branch?

[ralonsoh]

Hi @svinota. We can use https://review.opendev.org/c/openstack/neutron/+/942859, for example. In the script clone_and_install_pyroute2.sh it is possible to checkout a testing version or a patch proposed here. Let me know which one is. In any case, I've restored the patch and anyone can push a new PS.