gh-111784: Add PyCapsule_ImportCapsule and fix segfault in _elementtree.c

[Eclips4]

That's a PoC.

We can add a NEWS entry and doc about new public API later, if we consider this as the right solution.

Also, I was incorrect in issue. We need to keep reference to the capsule, not pyexpat module.

• Issue: Segfault during deallocation of _elementtree.XMLParser #111784

[Eclips4]

@mgorny can you confirm that this fixes the issue?

[Eclips4]

We don't need here Py_XDECREF(object) as in the PyCapsule_Import, because we do it later manually. So, we can control a process of capsule's deallocation

[mgorny]

@mgorny can you confirm that this fixes the issue?

I've applied it on top of 3.12.0 and I still see a segv with my reproducer from #111784.

[Eclips4]

@mgorny can you confirm that this fixes the issue?

I've applied it on top of 3.12.0 and I still see a segv with my reproducer from #111784.

Indeed. Interesting, reduced reproducer by @chgnrdv doesn't segfault'ed anymore, but this doesn't relevant for your example :(

[chgnrdv]

@Eclips4, it is reduced but not minimal :)
pyexpat/_elementtree has nothing to do with asyncio, and I guess that instantiation of _asyncio.Task just interferes with GC state (that may be invalid after _elementtree import) is some weakly deterministic way, which triggers segfault. We can't use this example as a stable way to reproduce the issue, and it could easily stop to work after some _asyncio changes being merged into main branch, for example.

[chgnrdv]

Other, a bit hacky repro:

import _elementtree
x = _elementtree.XMLParser()
import sys
del sys.modules['pyexpat']

Looks obvious in hindsight, as all what we need is to deallocate pyexpat capsule earlier than XMLParser instance.

[chgnrdv]

name_dup is always non-NULL here, as we return MemoryError at line 236 if PyMem_Malloc fails.

[Eclips4]

It's not a final version 😅
Currently we just talking about idea, sadly, but this solution is incorrect :(
FYI, it's just copied from PyCapsule_Import

[Eclips4]

Other, a bit hacky repro:

import _elementtree
x = _elementtree.XMLParser()
import sys
del sys.modules['pyexpat']

Looks obvious in hindsight, as all what we need is to deallocate pyexpat capsule earlier than XMLParser instance.

Yep, I thinking about the same.
However.. this code doesn't segfault's on this patch..

[Eclips4]

This PR fixes only one segfault (there's two segfaults actually) and way to fix this segfault is really weird. So I'm closing this.