Segfault during deallocation of _elementtree.XMLParser
#111784
Comments
mgorny
added
the
type-crash
|
Cc: @kumaraditya303 |
|
Reduced example, just in case: import _asyncio
import _elementtree
async def _end_stream_wait():
await "foo"
parser = _elementtree.XMLParser()
_asyncio.Task(_end_stream_wait()) |
|
The problem is more complicated than it may seem at first glance. What's actually happening and why this code segfaults?
cpython/Modules/_elementtree.c Lines 3777 to 3785 in 931f443 Line 3784 (which actually means (st->expat_capi->ParserFree)(parser)) leads to segfault if the second point (which described below) already executed.2. pyexpat_capsule trying to deallocate himself (st->expat_capi of _elementtree is pointing at this data!)
Where's the problem? Order of execution of two points described above is not strict. Second point can be executed before first ponit and vice-versa (which is okay for us). Situation when cc @vstinner |
Eclips4
changed the title
_elementtree (#104561)"_elementtree.XMLParser
|
|
Yes, that is what we should do. But I don't understand how we can make sure that the |
|
Can we keep a strong reference to the pyexpat module? Is it enough? |
There's no such API, and yes, it will be enough. |
|
To be honest, I don't see any solution without changing some code of capsule's implementation.. |
That's acceptable and has been done recently: see commit 513c89d. |
|
That's a kinda hard issue 😄 Why does it happens? There's a one interesting thing: if (m->md_state != NULL)
PyMem_Free(m->md_state);It's "clears" the state of module, if it's isn't freed previously. That's incorrect, because there is no guarantee that module doesn't use state to do some cleanup things in his deallocators. I'll take an opportunity to add here |
|
I wrote a PR, please check #113405 |
Seems you misunderstand me. Extension module doesn't tries to free his state. |
First fix resolve situation when pyexpat module (which contains expat_CAPI capsule) deallocates before _elementtree, so we need to hold a strong reference to pyexpat module to. Second fix resolve situation when module state is deallocated before deallocation of XMLParser instances, which uses module state to clear some stuff.
…H-113405) First fix resolve situation when pyexpat module (which contains expat_CAPI capsule) deallocates before _elementtree, so we need to hold a strong reference to pyexpat module to. Second fix resolve situation when module state is deallocated before deallocation of XMLParser instances, which uses module state to clear some stuff. (cherry picked from commit 894f0e5) Co-authored-by: Kirill Podoprigora <[email protected]>
) (GH-113446) First fix resolve situation when pyexpat module (which contains expat_CAPI capsule) deallocates before _elementtree, so we need to hold a strong reference to pyexpat module to. Second fix resolve situation when module state is deallocated before deallocation of XMLParser instances, which uses module state to clear some stuff. (cherry picked from commit 894f0e5) Co-authored-by: Kirill Podoprigora <[email protected]>
|
Thanks @mgorny for the report! |
github-project-automation
bot
moved this from Todo
to Done
in Release and Deferred blockers 🚫
|
Thanks a lot! |
…H-113405) First fix resolve situation when pyexpat module (which contains expat_CAPI capsule) deallocates before _elementtree, so we need to hold a strong reference to pyexpat module to. Second fix resolve situation when module state is deallocated before deallocation of XMLParser instances, which uses module state to clear some stuff.
…H-113405) First fix resolve situation when pyexpat module (which contains expat_CAPI capsule) deallocates before _elementtree, so we need to hold a strong reference to pyexpat module to. Second fix resolve situation when module state is deallocated before deallocation of XMLParser instances, which uses module state to clear some stuff.
…H-113405) First fix resolve situation when pyexpat module (which contains expat_CAPI capsule) deallocates before _elementtree, so we need to hold a strong reference to pyexpat module to. Second fix resolve situation when module state is deallocated before deallocation of XMLParser instances, which uses module state to clear some stuff.
…H-113405) First fix resolve situation when pyexpat module (which contains expat_CAPI capsule) deallocates before _elementtree, so we need to hold a strong reference to pyexpat module to. Second fix resolve situation when module state is deallocated before deallocation of XMLParser instances, which uses module state to clear some stuff.
Uh oh!
There was an error while loading. Please reload this page.
Crash report
What happened?
I've originally hit it while running slixmpp's test suite. I've been able to reduce it to the following program:
I've been able to reproduce this with 3.12.0, and the tips of 3.12 and main branches. I've been able to bisect it to the following commit:
Backtrace:
CC @kumaraditya303
CPython versions tested on:
3.12, CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.13.0a1+ (heads/main:ba8aa1fd37, Nov 6 2023, 16:26:31) [GCC 13.2.1 20231014]
Linked PRs
PyCapsule_ImportCapsuleand fix segfault in_elementtree.c#112053The text was updated successfully, but these errors were encountered: